2435e9f276925d2f64839a38eee9cae9af6c872a
Part og hugging models evaluated.

 

challenge
Searching for Legal Clauses by Analogy. Few-shot Contract Discovery Shared Task
submitter
p/tlen
submitted
2019-11-09 22:03:06.17514 UTC
original repo
ssh://gitolite@gonito.net/filipg/contract-discovery / branch hugging
publicly available at
git://gonito.net/contract-discovery / branch submission-04964
browsable at
https://gonito.net/gitlist/contract-discovery.git/submission-04964/
clone by
git clone --single-branch git://gonito.net/contract-discovery -b submission-04964
# model dev-0 Soft-F1.0 test-A Soft-F1.0
197 roberta-large-mnli 0.04368 0.05079
196 gpt2-medium 0.10427 0.11191
195 bert-large-uncased-whole-word-masking-finetuned-squad 0.16730 0.14378
193 bert-large-uncased 0.20710 0.18295
192 bert-large-cased 0.21066 0.20522
191 distilgpt2 0.21616 0.21090
174 roberta-large 0.31276 0.30945
172 bert-large-cased-whole-word-masking 0.31221 0.31181
166 bert-large-uncased-whole-word-masking 0.33703 0.34261
157 openai-gpt 0.37478 0.36546
155 xlm-mlm-en-2048 0.38203 0.37489
151 gpt2-large 0.41839 0.40711
146 openai-gpt-edgar-35-checkpoint-4000 0.44469 0.43041
144 openai-gpt-edgar-25-checkpoint-4000 0.44259 0.43102
Parameter Value
segm sentence
vectorizer hugging