4361e0138ca7222e7e04a4f27f6bbf1d99091e4a
flan-t5-large with language code extra context
- challenge
- CAICCAIC: Centre for Artificial Intelligence Challenge on Conversational AI Correctness
- submitter
- ked
- submitted
- 2023-06-08 18:05:43.332247 UTC
- file basename
- out
test-A / 308ab14207398c787520672094d6bfc8229fcdad
| Metric | Score |
|---|---|
| Exact Match Accuracy | 0.6652828731415331 |
| Domain accuracy | 0.954527901139216 |
| Intent accuracy | 0.8975670978953466 |
| Slot WRR | 0.7700253564451095 |
| en-US | es-ES | pl-PL | |
|---|---|---|---|
| Exact Match Accuracy | 0.5481459330143541 | 0.7940340909090909 | 0.6476817401259302 |
| Domain accuracy | 0.9013157894736842 | 0.9897727272727272 | 0.9699484831139096 |
| Intent accuracy | 0.8104066985645934 | 0.9573863636363636 | 0.9207212364052662 |
| Slot WRR | 0.36531141868512107 | 0.9235342540931126 | 0.8600217864923747 |