28eafc9c4595add84f83b5e2d7aca93ab5860828
flan-t5-base huggingface-transformers t5
- challenge
- CAICCAIC: Centre for Artificial Intelligence Challenge on Conversational AI Correctness
- submitter
- s478839
- submitted
- 2023-06-10 16:37:50.65035 UTC
- file basename
- out
test-A / e44c1a6f084bb0145e0422f5c87e3118c9aa9742
| Metric | Score |
|---|---|
| Exact Match Accuracy | 9.692990924888975e-2 |
| Domain accuracy | 0.9454527901139216 |
| Intent accuracy | 0.7097895346592006 |
| Slot WRR | 7.340898698923393e-2 |
| en-US | es-ES | pl-PL | |
|---|---|---|---|
| Exact Match Accuracy | 0.1507177033492823 | 5.909090909090909e-2 | 8.35718374356039e-2 |
| Domain accuracy | 0.8845693779904307 | 0.9860795454545455 | 0.9627933600457927 |
| Intent accuracy | 0.6220095693779905 | 0.7204545454545455 | 0.7830566685746995 |
| Slot WRR | 8.866782006920415e-2 | 5.9132900393304655e-2 | 8.265250544662306e-2 |