350ebcc49aa6e32258df077c37c2ce3d5dcbc7cc
flan-t5-base huggingface-transformers t5
- challenge
- CAICCAIC: Centre for Artificial Intelligence Challenge on Conversational AI Correctness
- submitter
- s478839
- submitted
- 2023-06-11 21:22:14.592982 UTC
- original repo
https://git.wmi.amu.edu.pl/s478839/cnlps-caiccaic/ branchmaster- publicly available at
git://gonito.net/cnlps-caiccaic/ branchsubmission-08856- browsable at
- https://gonito.net/gitlist/cnlps-caiccaic.git/submission-08856/
- clone by
git clone --single-branch git://gonito.net/cnlps-caiccaic -b submission-08856- file basename
- out
test-A / 48a284ff728e2113c80b6657124e6ce62ee5b7b4
| Metric | Score |
|---|---|
| Exact Match Accuracy | 0.4431357404904422 |
| Domain accuracy | 0.9449700714423633 |
| Intent accuracy | 0.8494883182081483 |
| Slot WRR | 0.6239140374942844 |
| en-US | es-ES | pl-PL | |
|---|---|---|---|
| Exact Match Accuracy | 0.389055023923445 | 0.4747159090909091 | 0.46307956496851743 |
| Domain accuracy | 0.8833732057416268 | 0.9840909090909091 | 0.9645105895821409 |
| Intent accuracy | 0.7511961722488039 | 0.9306818181818182 | 0.8617630223239839 |
| Slot WRR | 0.2433391003460208 | 0.7563797676758438 | 0.7262391067538126 |