CAICCAIC: Centre for Artificial Intelligence Challenge on Conversational AI Correctness
Develop Natural Language Understanding models that are robust to speech recognition errors. [ver. 1.0.13]
Deadline: 2023-06-22 12:01:00 UTC
This is a long list of all submissions, if you want to see only the best, click leaderboard.
| # | submitter | when | ver. | description | dev-A pl-PL Slot WRR | dev-A pl-PL Intent accuracy | dev-A pl-PL Domain accuracy | dev-A pl-PL Exact Match Accuracy | dev-A es-ES Slot WRR | dev-A es-ES Intent accuracy | dev-A es-ES Domain accuracy | dev-A es-ES Exact Match Accuracy | dev-A en-US Slot WRR | dev-A en-US Intent accuracy | dev-A en-US Domain accuracy | dev-A en-US Exact Match Accuracy | dev-A Slot WRR | dev-A Intent accuracy | dev-A Domain accuracy | dev-A Exact Match Accuracy | test-A pl-PL Slot WRR | test-A pl-PL Intent accuracy | test-A pl-PL Domain accuracy | test-A pl-PL Exact Match Accuracy | test-A es-ES Slot WRR | test-A es-ES Intent accuracy | test-A es-ES Domain accuracy | test-A es-ES Exact Match Accuracy | test-A en-US Slot WRR | test-A en-US Intent accuracy | test-A en-US Domain accuracy | test-A en-US Exact Match Accuracy | test-A Slot WRR | test-A Intent accuracy | test-A Domain accuracy | test-A Exact Match Accuracy | test-B pl-PL Slot WRR | test-B pl-PL Intent accuracy | test-B pl-PL Domain accuracy | test-B pl-PL Exact Match Accuracy | test-B es-ES Slot WRR | test-B es-ES Intent accuracy | test-B es-ES Domain accuracy | test-B es-ES Exact Match Accuracy | test-B en-US Slot WRR | test-B en-US Intent accuracy | test-B en-US Domain accuracy | test-B en-US Exact Match Accuracy | test-B Slot WRR | test-B Intent accuracy | test-B Domain accuracy | test-B Exact Match Accuracy | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 30 | Martyna Druminska | 2023-06-12 21:47 | 1.0.12 | dfd | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.6911083877995643 | 0.7421293646250715 | 0.9768174012593016 | 0.4693760732684602 | 0.6935882191530229 | 0.6821022727272728 | 0.9894886363636364 | 0.45681818181818185 | 0.4185121107266436 | 0.59688995215311 | 0.9084928229665071 | 0.41088516746411485 | 0.6267406576048551 | 0.6748407028383858 | 0.9590654566518633 | 0.44622513998841473 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 29 | Jakub Adamski | 2023-06-11 23:01 | 1.0.12 | flan-t5 fine-tuning | 2.994052446607187e-2 | 1.8296169239565466e-2 | 2.572898799313894e-3 | 0.0 | 1.8653236336504397e-2 | 2.3406655386350818e-2 | 3.6661026508742244e-3 | 0.0 | 4.5936844780481656e-2 | 6.496008808147537e-2 | 8.257638315441783e-4 | 0.0 | 3.012755151118718e-2 | 3.587149948487403e-2 | 2.341481689613187e-3 | 0.0 | 2.689270152505452e-2 | 1.8317115054378934e-2 | 5.724098454493417e-4 | 0.0 | 1.815604134272386e-2 | 2.3295454545454546e-2 | 5.965909090909091e-3 | 0.0 | 0.1166089965397924 | 3.738038277511962e-2 | 2.6913875598086126e-3 | 0.0 | 4.447769879868646e-2 | 2.61633519984553e-2 | 3.0893994979725816e-3 | 0.0 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 28 | Jakub Adamski | 2023-06-11 21:59 | 1.0.12 | flan-t5 | 1.4666125979994615e-2 | 2.2298456260720412e-2 | 2.001143510577473e-3 | 1.7152658662092624e-3 | 1.2264502891251583e-2 | 2.763677382966723e-2 | 2.8200789622109417e-3 | 1.1280315848843769e-3 | 3.3102484274731614e-2 | 7.074043490228461e-2 | 2.477291494632535e-3 | 2.752546105147261e-4 | 1.9257777179245483e-2 | 4.05544628641004e-2 | 2.435140957197715e-3 | 1.0302519434298024e-3 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 27 | s478839 | 2023-06-11 21:22 | 1.0.12 | flan-t5-base huggingface-transformers t5 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.7262391067538126 | 0.8617630223239839 | 0.9645105895821409 | 0.46307956496851743 | 0.7563797676758438 | 0.9306818181818182 | 0.9840909090909091 | 0.4747159090909091 | 0.2433391003460208 | 0.7511961722488039 | 0.8833732057416268 | 0.389055023923445 | 0.6239140374942844 | 0.8494883182081483 | 0.9449700714423633 | 0.4431357404904422 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 26 | s478839 | 2023-06-11 21:08 | 1.0.12 | flan-t5-base huggingface-transformers t5 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.7262391067538126 | 0.8617630223239839 | 0.9645105895821409 | 0.46307956496851743 | N/A | N/A | N/A | N/A | 0.2433391003460208 | 0.7511961722488039 | 0.8833732057416268 | 0.389055023923445 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 25 | s478839 | 2023-06-11 20:43 | 1.0.12 | flan-t5-base huggingface-transformers t5 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.11989379084967322 | 0.7799084144247281 | 0.9645105895821409 | 8.900973096737264e-2 | 6.969724686728251e-2 | 0.7227272727272728 | 0.9840909090909091 | 7.130681818181818e-2 | 8.685121107266436e-2 | 0.6214114832535885 | 0.8833732057416268 | 0.14055023923444976 | 8.914245333998416e-2 | 0.7093068159876424 | 0.9449700714423633 | 9.963313380961576e-2 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 24 | s478873 | 2023-06-11 19:33 | 1.0.12 | s478873 transformer t5 | 0.7464179507975128 | 0.9316752429959977 | 0.9797026872498571 | 0.510863350485992 | 0.6339302368961015 | 0.835589396503102 | 0.9467005076142132 | 0.4010152284263959 | 0.7986530275112778 | 0.9499036608863198 | 0.985962014863749 | 0.6534544453619598 | 0.7158275457396256 | 0.9059660953451344 | 0.970871967781212 | 0.522899690924417 | 0.7578295206971677 | 0.9284487693188322 | 0.979965655409273 | 0.5160274756725816 | 0.6329918595079118 | 0.834659090909091 | 0.9443181818181818 | 0.38920454545454547 | 0.446280276816609 | 0.8343301435406698 | 0.9210526315789473 | 0.4805622009569378 | 0.626241842291225 | 0.8661903842440626 | 0.9488318208148291 | 0.4614790500096544 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 23 | s478873 | 2023-06-11 17:18 | 1.0.12 | s478873 transformer t5 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.7578295206971677 | 0.9284487693188322 | 0.979965655409273 | 0.5160274756725816 | 0.6329918595079118 | 0.834659090909091 | 0.9443181818181818 | 0.38920454545454547 | 0.446280276816609 | 0.8343301435406698 | 0.9210526315789473 | 0.4805622009569378 | 0.626241842291225 | 0.8661903842440626 | 0.9488318208148291 | 0.4614790500096544 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 17 | s444386 | 2023-06-11 15:44 | 1.0.12 | mbart large huggingface-transformers mbart-large-50 | 0.904771559881049 | 0.9588336192109777 | 0.9882790165809033 | 0.7924528301886793 | 0.9577504196978176 | 0.9737732656514383 | 0.9906937394247038 | 0.8877608573040046 | 0.8917974458351865 | 0.9540324800440407 | 0.9779796311588219 | 0.791907514450867 | 0.9226995517420497 | 0.9621616558958509 | 0.9855764727919828 | 0.8239205769410883 | 0.914692265795207 | 0.953921007441328 | 0.9854035489410418 | 0.7993703491700057 | 0.9555016921247599 | 0.9747159090909091 | 0.9928977272727273 | 0.8835227272727273 | 0.6580449826989619 | 0.8142942583732058 | 0.9078947368421053 | 0.569377990430622 | 0.8715758407116432 | 0.9159104074145588 | 0.962927206024329 | 0.7537169337709982 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 22 | s444415 | 2023-06-11 13:00 | 1.0.12 | flan t5 large v3 transformer t5 | 0.7637199243038659 | 0.7944539736992567 | 0.9325328759291024 | 0.4937106918238994 | 0.8799197910837531 | 0.8849407783417935 | 0.9692611393119007 | 0.6771009588268472 | 0.8075481288518965 | 0.8235617946600605 | 0.9622901183594825 | 0.6746490503715937 | 0.8249292983704958 | 0.8344104149105555 | 0.9548562330242577 | 0.6161843214386064 | 0.7610974945533769 | 0.7882083571837436 | 0.9235832856325129 | 0.49370349170005723 | 0.8678313363212293 | 0.8647727272727272 | 0.9681818181818181 | 0.6588068181818182 | 0.23053633217993075 | 0.659688995215311 | 0.853767942583732 | 0.36363636363636365 | 0.6821299413892006 | 0.772736049430392 | 0.9162000386174938 | 0.5078200424792431 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 21 | Mikołaj Pokrywka | 2023-06-11 05:47 | 1.0.12 | all done | 0.5625844822925115 | 0.6446540880503144 | 0.9493996569468267 | 0.3630646083476272 | 0.5940123111359821 | 0.6398759165256627 | 0.9774393683023125 | 0.39142695995487875 | 0.5406950886333313 | 0.6229011835948252 | 0.9749518304431599 | 0.43572804844481144 | 0.5689220646799669 | 0.635665449096188 | 0.9674065748805845 | 0.3972089538259811 | 0.5509259259259259 | 0.6199198626216371 | 0.9318832283915284 | 0.34430452203777906 | 0.5751852190615567 | 0.6264204545454546 | 0.975 | 0.3684659090909091 | 9.065743944636673e-2 | 0.4946172248803828 | 0.8690191387559809 | 0.27811004784689 | 0.4513655069210625 | 0.5816759992276501 | 0.9262405869859046 | 0.3311450086889361 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 20 | s443930 | 2023-06-10 22:14 | 1.0.12 | flan-t5-2nd-try | 0.7249932414165992 | 0.9205260148656375 | 0.9791309319611207 | 0.48770726129216696 | 0.7707050923335199 | 0.957980823463057 | 0.9887196841511562 | 0.5143824027072758 | 0.7856915941292331 | 0.9432975502339664 | 0.9834847233691164 | 0.6333608587943848 | 0.7622309009407645 | 0.9407136836189941 | 0.9837969467078768 | 0.5461271892853798 | 0.7393110021786493 | 0.9170005724098454 | 0.9762449914138523 | 0.4951345163136806 | 0.7690935699259124 | 0.9602272727272727 | 0.9897727272727272 | 0.503125 | 0.48425605536332184 | 0.8133971291866029 | 0.9061004784688995 | 0.479366028708134 | 0.6915658644053706 | 0.8982429040355281 | 0.9581965630430586 | 0.49275921992662675 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 19 | s478839 | 2023-06-10 16:37 | 1.0.12 | flan-t5-base huggingface-transformers t5 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 8.265250544662306e-2 | 0.7830566685746995 | 0.9627933600457927 | 8.35718374356039e-2 | 5.9132900393304655e-2 | 0.7204545454545455 | 0.9860795454545455 | 5.909090909090909e-2 | 8.866782006920415e-2 | 0.6220095693779905 | 0.8845693779904307 | 0.1507177033492823 | 7.340898698923393e-2 | 0.7097895346592006 | 0.9454527901139216 | 9.692990924888975e-2 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 18 | s443930 | 2023-06-10 13:48 | 1.0.12 | flan-t5 finetuned fine-tuned | 0.6906596377399297 | 0.8533447684391081 | 0.9611206403659234 | 0.43539165237278443 | 0.7478082447304608 | 0.928369994359842 | 0.9813874788494078 | 0.47828539199097575 | 0.7422962068746426 | 0.8918249380677127 | 0.973850812001101 | 0.5813377374071016 | 0.7298716789472672 | 0.8913552496019481 | 0.9721831975273953 | 0.49929755549311605 | 0.7068355119825709 | 0.8480251860331998 | 0.95220377790498 | 0.4390383514596451 | 0.7469587487423397 | 0.9261363636363636 | 0.9826704545454545 | 0.4661931818181818 | 0.3342560553633218 | 0.7511961722488039 | 0.8735047846889952 | 0.4061004784688995 | 0.6355530614789874 | 0.8433095192122031 | 0.9371500289631203 | 0.4376327476346785 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 15 | ked | 2023-06-08 18:05 | 1.0.12 | flan-t5-large with language code extra context | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.8600217864923747 | 0.9207212364052662 | 0.9699484831139096 | 0.6476817401259302 | 0.9235342540931126 | 0.9573863636363636 | 0.9897727272727272 | 0.7940340909090909 | 0.36531141868512107 | 0.8104066985645934 | 0.9013157894736842 | 0.5481459330143541 | 0.7700253564451095 | 0.8975670978953466 | 0.954527901139216 | 0.6652828731415331 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 14 | s444386 | 2023-06-07 18:56 | 1.0.12 | mbart-large transformer mbart-large-50 | 0.8586780210867802 | 0.9359634076615209 | 0.9802744425385935 | 0.6612349914236707 | 0.9300037306472673 | 0.9599548787366047 | 0.9887196841511562 | 0.8141567963902989 | 0.9188004320477794 | 0.9526562069914671 | 0.985962014863749 | 0.8830167905312414 | 0.9063083168202544 | 0.9496113140395243 | 0.9850145171864756 | 0.7874871218507071 | 0.914692265795207 | 0.953921007441328 | 0.9854035489410418 | 0.7993703491700057 | 0.9555016921247599 | 0.9747159090909091 | 0.9928977272727273 | 0.8835227272727273 | 0.6580449826989619 | 0.8142942583732058 | 0.9078947368421053 | 0.569377990430622 | 0.8715758407116432 | 0.9159104074145588 | 0.962927206024329 | 0.7537169337709982 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 16 | s444476 | 2023-06-07 18:22 | 1.0.12 | flanT5-large transformer t5 | 0.8387402000540687 | 0.9456832475700401 | 0.9842767295597484 | 0.6423670668953688 | 0.932288752098489 | 0.9672870840383531 | 0.9943598420755781 | 0.802312464749013 | 0.9292839443420802 | 0.9636663914120561 | 0.9895403248004404 | 0.8755849160473438 | 0.9047499951903654 | 0.958977240797977 | 0.9894165027629483 | 0.774843120726796 | 0.837962962962963 | 0.9358900973096738 | 0.979107040641099 | 0.6485403548941042 | 0.9023598280435379 | 0.9676136363636364 | 0.9928977272727273 | 0.7866477272727272 | 0.5785467128027681 | 0.8594497607655502 | 0.9327153110047847 | 0.6276913875598086 | 0.8049008604564161 | 0.9219926626761923 | 0.9688163738173392 | 0.6887430005792624 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 13 | s444386 | 2023-06-06 16:57 | 1.0.12 | Flan T5 Large text2text transformer t5 | 0.8586780210867802 | 0.9359634076615209 | 0.9802744425385935 | 0.6612349914236707 | 0.9300037306472673 | 0.9599548787366047 | 0.9887196841511562 | 0.8141567963902989 | 0.9188004320477794 | 0.9526562069914671 | 0.985962014863749 | 0.8830167905312414 | 0.9063083168202544 | 0.9496113140395243 | 0.9850145171864756 | 0.7874871218507071 | 0.8684640522875817 | 0.934745277618775 | 0.9776760160274757 | 0.6648540354894105 | 0.9321320771974755 | 0.9644886363636364 | 0.9920454545454546 | 0.8071022727272728 | 0.4777681660899654 | 0.8110047846889952 | 0.902811004784689 | 0.5750598086124402 | 0.8035291183439331 | 0.9049044217030314 | 0.9583896505116818 | 0.6842054450666152 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 12 | s444476 | 2023-06-06 12:55 | 1.0.12 | flanT5-base transformer t5 | 0.7176263855095972 | 0.7504288164665524 | 0.9276729559748428 | 0.43567753001715265 | 0.8539451594851707 | 0.8544839255499154 | 0.9636209813874789 | 0.6156232374506486 | 0.7943325497172629 | 0.782824112303881 | 0.9534819708230113 | 0.6215249105422516 | 0.7970911329575405 | 0.7960101152008991 | 0.9483937435609253 | 0.5586775311417065 | 0.7242647058823529 | 0.7335432169433315 | 0.9129937034917001 | 0.42472810532341154 | 0.8542943382420196 | 0.8590909090909091 | 0.9647727272727272 | 0.6102272727272727 | 0.31496539792387546 | 0.6369617224880383 | 0.8343301435406698 | 0.4019138755980861 | 0.6850189134139751 | 0.7450279976829504 | 0.9051940529059664 | 0.48040162193473646 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 11 | s444476 | 2023-06-06 12:39 | 1.0.12 | flanT5-base transformer t5 | 0.1405109489051095 | 0.6849628359062321 | 0.9271012006861064 | 6.975414522584333e-2 | 9.499160604364854e-2 | 0.7007896221094191 | 0.9630569655950366 | 6.2887760857304e-2 | 0.12859775081008962 | 0.7013487475915221 | 0.9587118084227911 | 0.1434076520781723 | 0.11812462725331385 | 0.6957946988854548 | 0.9497986325746932 | 9.253535637351316e-2 | 0.13657407407407407 | 0.6685746994848312 | 0.9149971379507728 | 7.183743560389239e-2 | 9.599378029817984e-2 | 0.7045454545454546 | 0.9642045454545455 | 6.875e-2 | 9.333910034602078e-2 | 0.5523325358851675 | 0.8379186602870813 | 0.13217703349282298 | 0.1077441077441077 | 0.6432709017184784 | 0.9068352963892643 | 9.026839158138637e-2 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 10 | s444476 | 2023-06-06 12:25 | 1.0.12 | flanT5-base transformer t5 | 2.2438496891051596e-2 | 0.6849628359062321 | 0.9271012006861064 | 0.0 | 3.6420443947024816e-2 | 0.7007896221094191 | 0.9630569655950366 | 0.0 | 2.236482622784164e-2 | 0.7013487475915221 | 0.9587118084227911 | 1.3762730525736307e-3 | 2.8184459108486104e-2 | 0.6957946988854548 | 0.9497986325746932 | 4.6829633792263745e-4 | 2.2126906318082784e-2 | 0.6685746994848312 | 0.9149971379507728 | 0.0 | 3.631208268544772e-2 | 0.7045454545454546 | 0.9642045454545455 | 0.0 | -8.217993079584707e-3 | 0.5523325358851675 | 0.8379186602870813 | 2.15311004784689e-2 | 2.1282786714885482e-2 | 0.6432709017184784 | 0.9068352963892643 | 6.951148870438309e-3 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 9 | s444476 | 2023-06-06 12:19 | 1.0.12 | flanT5-base transformer t5 | 2.2303325223033243e-2 | 0.6778158947970269 | 0.9271012006861064 | 0.0 | 3.6420443947024816e-2 | 0.643824027072758 | 0.9630569655950366 | 0.0 | 2.115763390304337e-2 | 0.6650151390035783 | 0.9584365538122763 | 0.0 | 2.7780449797033424e-2 | 0.6621710218226093 | 0.9497049733071088 | 0.0 | 2.19907407407407e-2 | 0.6625643961076131 | 0.9149971379507728 | 0.0 | 3.6266349583828794e-2 | 0.6536931818181818 | 0.9642045454545455 | 0.0 | -2.9584775086505255e-2 | 0.5167464114832536 | 0.8379186602870813 | 0.0 | 1.608679386457168e-2 | 0.6124734504730643 | 0.9068352963892643 | 0.0 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 8 | s444476 | 2023-06-06 11:57 | 1.0.12 | flanT5-base transformer t5 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 7 | s444415 | 2023-06-05 07:54 | 1.0.12 | flan t5 large v2 transformer t5 | 0.800081103000811 | 0.8553459119496856 | 0.9542595769010863 | 0.5543167524299599 | 0.8996455885096064 | 0.9128595600676819 | 0.9774393683023125 | 0.7143260011280316 | 0.8543109473282928 | 0.8678777869529315 | 0.9694467382328654 | 0.740710156895128 | 0.8575770984436022 | 0.8787112484780369 | 0.9671255970778309 | 0.6708813337079704 | 0.8154956427015251 | 0.8548941041785919 | 0.95220377790498 | 0.5629650829994276 | 0.8888685630659472 | 0.9102272727272728 | 0.9792613636363636 | 0.6991477272727272 | 0.34688581314878897 | 0.7081339712918661 | 0.8630382775119617 | 0.4339114832535885 | 0.7362514029180696 | 0.8263178219733539 | 0.932612473450473 | 0.5675806140181502 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 6 | s444415 | 2023-06-03 18:15 | 1.0.12 | Flan-t5-small transformer small t5 | 0.2511489591781563 | 0.30131503716409375 | 0.6532304173813608 | 7.375643224699828e-2 | 0.5093266181682522 | 0.46841511562323745 | 0.853355893965031 | 0.1897913141567964 | 0.3797572908062774 | 0.37462152491054224 | 0.8329204514175612 | 0.11395540875309662 | 0.3966024740760692 | 0.38175517467453407 | 0.7808373138522057 | 0.12597171490118947 | 0.2580337690631809 | 0.28906697195191755 | 0.6396680022896394 | 7.326846021751574e-2 | 0.5074544955638891 | 0.47244318181818185 | 0.846875 | 0.18693181818181817 | -9.359861591695506e-2 | 0.3104066985645933 | 0.7090311004784688 | 5.651913875598086e-2 | 0.2869019412229289 | 0.3582737980305078 | 0.7324773122224367 | 0.10648773894574243 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 5 | s444386 | 2023-06-03 16:59 | 1.0.12 | flan t5 large transformer | 0.7860908353609084 | 0.8141795311606632 | 0.9453973699256718 | 0.5268724985706118 | 0.8961014736056706 | 0.9086294416243654 | 0.9771573604060914 | 0.7306824591088551 | 0.8494821780290998 | 0.8420038535645472 | 0.9697219928433801 | 0.7374071015689513 | 0.8506704630716251 | 0.8550154537791514 | 0.9642221597827105 | 0.6661983703287441 | 0.8684640522875817 | 0.934745277618775 | 0.9776760160274757 | 0.6648540354894105 | 0.9321320771974755 | 0.9644886363636364 | 0.9920454545454546 | 0.8071022727272728 | 0.4777681660899654 | 0.8110047846889952 | 0.902811004784689 | 0.5750598086124402 | 0.8035291183439331 | 0.9049044217030314 | 0.9583896505116818 | 0.6842054450666152 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 3 | s444415 | 2023-06-03 16:39 | 1.0.12 | Flan-t5-large transformer t5 | 0.8047445255474452 | 0.8599199542595769 | 0.9608347627215552 | 0.5566037735849056 | 0.9009513150531617 | 0.9196277495769881 | 0.9816694867456289 | 0.724478285391991 | 0.846813647626914 | 0.8665015139003578 | 0.9686209744013212 | 0.7390586292320397 | 0.8571730891321495 | 0.8819893228434954 | 0.9704036714432893 | 0.6744403858761825 | 0.8029684095860566 | 0.8388666285060103 | 0.9484831139095592 | 0.5495134516313681 | 0.8979237171864995 | 0.9142045454545454 | 0.9789772727272728 | 0.7161931818181818 | 0.352681660899654 | 0.7078349282296651 | 0.8621411483253588 | 0.4354066985645933 | 0.7379349046015713 | 0.8221664413979533 | 0.9309712299671752 | 0.5693184012357598 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 4 | s444386 | 2023-06-02 18:23 | 1.0.12 | flan-t5-base transformer | 0.7860908353609084 | 0.8141795311606632 | 0.9453973699256718 | 0.5268724985706118 | 0.8961014736056706 | 0.9086294416243654 | 0.9771573604060914 | 0.7306824591088551 | 0.8494821780290998 | 0.8420038535645472 | 0.9697219928433801 | 0.7374071015689513 | 0.8506704630716251 | 0.8550154537791514 | 0.9642221597827105 | 0.6661983703287441 | 0.7946623093681917 | 0.8062392673153979 | 0.9416141957641672 | 0.5263308528906697 | 0.8906064209274673 | 0.9051136363636364 | 0.9795454545454545 | 0.7127840909090909 | 0.2604671280276817 | 0.7114234449760766 | 0.8654306220095693 | 0.4386961722488038 | 0.7099181111526791 | 0.809229581000193 | 0.929909248889747 | 0.5614018150222051 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 2 | Tomasz Ziętkiewicz | 2023-04-25 09:35 | 1.0.5 | Official baseline prepared by task authors. See task description for details. baseline roberta-xlm | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.8099809368191722 | 0.9593589009730967 | 0.9896966227819118 | 0.7670291929021179 | 0.7232690021037227 | 0.9707386363636363 | 0.9954545454545455 | 0.5948863636363636 | 0.733477508650519 | 0.9040071770334929 | 0.9545454545454546 | 0.6860047846889952 | 0.7521927089828324 | 0.9453562463796099 | 0.9803050782004248 | 0.682371114114694 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 1 | [anonymized] | 2023-04-24 14:35 | 1.0.5 | Official baseline prepared by organizers | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.8099809368191722 | 0.9593589009730967 | 0.9896966227819118 | 0.7670291929021179 | 0.7232690021037227 | 0.9707386363636363 | 0.9954545454545455 | 0.5948863636363636 | 0.7333044982698962 | 0.9040071770334929 | 0.9545454545454546 | 0.6857057416267942 | 0.7521511410400299 | 0.9453562463796099 | 0.9803050782004248 | 0.6822745703803823 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |