d5f1ea019fa9c255f0c56b4ae8f6a319a1d4d125
Rename

 

challenge
Searching for Legal Clauses by Analogy. Few-shot Contract Discovery Shared Task
submitter
p/tlen
submitted
2019-11-09 21:32:51.095589 UTC
original repo
ssh://gitolite@gonito.net/filipg/contract-discovery / branch wmd
publicly available at
git://gonito.net/contract-discovery / branch submission-04960
browsable at
https://gonito.net/gitlist/contract-discovery.git/submission-04960/
clone by
git clone --single-branch git://gonito.net/contract-discovery -b submission-04960
file basename
out-distance=wmd,model=lmmodel,segm=sentence,chr=Kbest-1
Parameter Value
chr Kbest-1
distance wmd
model lmmodel
segm sentence

test-A / 4cf349264dc164cc0482d50b7adb30d5a99ccc4e
Metric Score
Soft-F1.0 0.34501

dev-0 / 5747c80fe8ba39737a149cd7d0249c5da9de96a4
Metric Score
Soft-F1.0 0.37088

worst items

# input expected output actual output test-A Soft-F1.0
1 CBN_152 change_of_control_covenant CBN_110 78189-79110 change_of_control_covenant:48096-48789 change_of_control_covenant:22865-23246 0.00000
2 CBN_168 change_of_control_covenant CBN_064 139148-139417 change_of_control_covenant:94644-95435 change_of_control_covenant:28785-29201 0.00000
3 CBN_134 change_of_control_covenant CBN_018 134560-135029 CBN_076 57245-57487,58481-58694 CBN_182 67446-68304 CBN_110 78189-79110 CBN_028 627… change_of_control_covenant:71022-71487 change_of_control_covenant:37409-37702 0.00000
4 CBN_046 change_of_control_covenant CBN_200 120080-121152 CBN_042 129116-129856 CBN_180 150508-151058 CBN_078 315437-316270 CBN_194 63017-635… change_of_control_covenant:211233-211471 change_of_control_covenant:35893-36258 0.00000
5 CBN_086 change_of_control_covenant CBN_120 373976-374706 CBN_066 94837-95722 change_of_control_covenant:93475-94326,79382-79820 change_of_control_covenant:2993-4509 0.00000
6 CBN_068 change_of_control_covenant CBN_076 57245-57487,58481-58694 change_of_control_covenant:90207-91323 change_of_control_covenant:1739-2003 0.00000
7 CBN_032 change_of_control_covenant CBN_144 155550-156564 CBN_068 90207-91323 CBN_014 41047-41622 CBN_056 107433-108274 change_of_control_covenant:85175-85924 change_of_control_covenant:26314-27056 0.00000
8 CBN_030 change_of_control_covenant CBN_004 49742-50425 change_of_control_covenant:257062-257455 change_of_control_covenant:93282-94377 0.00000
9 CBN_032 change_of_control_covenant CBN_146 148412-149022 CBN_144 155550-156564 CBN_076 57245-57487,58481-58694 CBN_090 262690-262969 change_of_control_covenant:85175-85924 change_of_control_covenant:252839-253447 0.00000
10 CBN_068 change_of_control_covenant CBN_204 55501-56158 CBN_034 132944-133574 change_of_control_covenant:90207-91323 change_of_control_covenant:95705-97155 0.00000
11 CBN_090 change_of_control_covenant CBN_146 148412-149022 CBN_062 21635-22393 change_of_control_covenant:262690-262969 change_of_control_covenant:267954-268430 0.00000
12 CBN_078 change_of_control_covenant CBN_198 35168-35886 CBN_132 5243-6123 CBN_104 174377-175177 change_of_control_covenant:315437-316270 change_of_control_covenant:122071-122677 0.00000
13 CBN_032 change_of_control_covenant CBN_064 139148-139417 CBN_144 155550-156564 CBN_110 78189-79110 CBN_104 174377-175177 CBN_056 107433-1082… change_of_control_covenant:85175-85924 change_of_control_covenant:26314-27056 0.00000
14 CBN_150 change_of_control_covenant CBN_064 139148-139417 change_of_control_covenant:26227-26985 change_of_control_covenant:22151-22411 0.00000
15 CBN_100 change_of_control_covenant CBN_014 41047-41622 CBN_056 107433-108274 change_of_control_covenant:97014-97652 change_of_control_covenant:79465-79802 0.00000
16 CBN_046 change_of_control_covenant CBN_064 139148-139417 CBN_042 129116-129856 CBN_132 5243-6123 CBN_034 132944-133574 change_of_control_covenant:211233-211471 change_of_control_covenant:35893-36258 0.00000
17 CBN_032 change_of_control_covenant CBN_198 35168-35886 CBN_192 42556-43160 CBN_078 315437-316270 CBN_090 262690-262969 CBN_066 94837-95722 change_of_control_covenant:85175-85924 change_of_control_covenant:252839-253447 0.00000
18 CBN_030 change_of_control_covenant CBN_070 219239-219671,345854-346289 CBN_066 94837-95722 CBN_028 62708-63411 CBN_150 26227-26985 CBN_082 4… change_of_control_covenant:257062-257455 change_of_control_covenant:32754-33858 0.00000
19 CBN_134 change_of_control_covenant CBN_030 257062-257455 CBN_068 90207-91323 change_of_control_covenant:71022-71487 change_of_control_covenant:73231-74520 0.00000
20 CBN_110 change_of_control_covenant CBN_144 155550-156564 CBN_078 315437-316270 CBN_066 94837-95722 CBN_014 41047-41622 change_of_control_covenant:78189-79110 change_of_control_covenant:2652-4097 0.00000
21 CBN_064 change_of_control_covenant CBN_058 55606-56287 CBN_110 78189-79110 CBN_014 41047-41622 CBN_210 63944-65061 change_of_control_covenant:139148-139417 change_of_control_covenant:139834-140407 0.00000
22 CBN_032 change_of_control_covenant CBN_180 150508-151058 CBN_024 48900-49547 CBN_078 315437-316270 CBN_028 62708-63411 change_of_control_covenant:85175-85924 change_of_control_covenant:252839-253447 0.00000
23 CBN_152 change_of_control_covenant CBN_058 55606-56287 CBN_192 42556-43160 CBN_068 90207-91323 CBN_188 45659-46119 change_of_control_covenant:48096-48789 change_of_control_covenant:23247-23670 0.00000
24 CBN_070 change_of_control_covenant CBN_192 42556-43160 CBN_100 97014-97652 CBN_104 174377-175177 change_of_control_covenant:219239-219671,345854-346289 change_of_control_covenant:14997-15415 0.00000
25 CBN_090 change_of_control_covenant CBN_014 41047-41622 CBN_004 49742-50425 change_of_control_covenant:262690-262969 change_of_control_covenant:264512-266354 0.00000
26 CBN_022 change_of_control_covenant CBN_086 93475-94326,79382-79820 CBN_076 57245-57487,58481-58694 CBN_094 140650-141452 CBN_034 132944-1335… change_of_control_covenant:165293-166211 change_of_control_covenant:90363-90910 0.00000
27 CBN_032 change_of_control_covenant CBN_014 41047-41622 change_of_control_covenant:85175-85924 change_of_control_covenant:26314-27056 0.00000
28 CBN_032 change_of_control_covenant CBN_026 44097-44819 CBN_188 45659-46119 CBN_182 67446-68304 CBN_094 140650-141452 CBN_014 41047-41622 change_of_control_covenant:85175-85924 change_of_control_covenant:252839-253447 0.00000
29 CBN_022 change_of_control_covenant CBN_026 44097-44819 CBN_064 139148-139417 CBN_100 97014-97652 CBN_094 140650-141452 CBN_066 94837-95722 change_of_control_covenant:165293-166211 change_of_control_covenant:121680-122008 0.00000
30 CBN_070 change_of_control_covenant CBN_068 90207-91323 CBN_104 174377-175177 change_of_control_covenant:219239-219671,345854-346289 change_of_control_covenant:14997-15415 0.00000
31 CBN_022 change_of_control_covenant CBN_192 42556-43160 CBN_168 94644-95435 CBN_182 67446-68304 CBN_014 41047-41622 CBN_082 49653-50153 change_of_control_covenant:165293-166211 change_of_control_covenant:356071-357022 0.00000
32 CBN_030 change_of_control_covenant CBN_032 85175-85924 CBN_146 148412-149022 CBN_194 63017-63511 change_of_control_covenant:257062-257455 change_of_control_covenant:65547-66590 0.00000
33 CBN_134 change_of_control_covenant CBN_064 139148-139417 CBN_086 93475-94326,79382-79820 CBN_030 257062-257455 CBN_034 132944-133574 change_of_control_covenant:71022-71487 change_of_control_covenant:37409-37702 0.00000
34 CBN_026 change_of_control_covenant CBN_024 48900-49547 CBN_066 94837-95722 change_of_control_covenant:44097-44819 change_of_control_covenant:31684-32051 0.00000
35 CBN_022 change_of_control_covenant CBN_064 139148-139417 CBN_086 93475-94326,79382-79820 CBN_150 26227-26985 CBN_082 49653-50153 change_of_control_covenant:165293-166211 change_of_control_covenant:90363-90910 0.00000
36 CBN_144 change_of_control_covenant CBN_022 165293-166211 change_of_control_covenant:155550-156564 change_of_control_covenant:156566-157220 0.00000
37 CBN_028 change_of_control_covenant CBN_026 44097-44819 CBN_082 49653-50153 change_of_control_covenant:62708-63411 change_of_control_covenant:41477-41817 0.00000
38 CBN_100 change_of_control_covenant CBN_064 139148-139417 CBN_192 42556-43160 change_of_control_covenant:97014-97652 change_of_control_covenant:79465-79802 0.00000
39 CBN_076 change_of_control_covenant CBN_078 315437-316270 change_of_control_covenant:57245-57487,58481-58694 change_of_control_covenant:102047-103341 0.00000
40 CBN_022 change_of_control_covenant CBN_018 134560-135029 CBN_086 93475-94326,79382-79820 CBN_068 90207-91323 CBN_082 49653-50153 change_of_control_covenant:165293-166211 change_of_control_covenant:90363-90910 0.00000

Compare with other submission