Transformer encoder-decoder models (seq2seq) transformer huggingface-transformers transformer-encoder-decoder seq2seq
- challenge
- GLUE-LM-GAP
- submitter
- kaczla
- submitted
- 2023-03-06 15:48:25.968755 UTC
# |
model_name |
dev-0 PerplexityHashed |
test-A PerplexityHashed |
|
152 |
LongT5-Local-base |
1006246.726444
|
1008375.972086
|
|
130 |
ByT5-base |
854668.733905
|
855967.364657
|
|
125 |
ByT5-small |
577506.468555
|
584584.351424
|
|
124 |
FLAN-T5-large |
544241.485156
|
562529.862349
|
|
123 |
T5-efficient-mini |
493098.539241
|
510519.379968
|
|
117 |
T5-efficient-tiny |
408282.416232
|
418979.060903
|
|
116 |
T5-large-v1_1 |
420124.898821
|
415089.099066
|
|
112 |
FLAN-T5-small |
306911.104213
|
324604.408047
|
|
109 |
T5-large-v1_1-lm-adapt |
337413.972666
|
300663.100084
|
|
108 |
T5-small-v1_1 |
290626.365308
|
282303.546243
|
|
107 |
T5-efficient-small |
303742.494167
|
277887.449243
|
|
106 |
T5-base-v1_1 |
290005.373172
|
267972.646440
|
|
102 |
T5-small-v1_1-lm-adapt |
239393.520998
|
233990.717024
|
|
101 |
LongT5-TGlobal-base |
199552.610743
|
211414.917005
|
|
96 |
T5-efficient-base |
152750.729495
|
124430.988625
|
|
95 |
FLAN-T5-base |
103892.908235
|
118104.568290
|
|
94 |
T5-efficient-large |
105072.298767
|
84932.099781
|
|
92 |
T5-base-v1_1-lm-adapt |
74652.507386
|
63737.130182
|
|
61 |
mT5-small |
4691.781349
|
3976.321841
|
|
55 |
mT5-base |
3046.209545
|
2491.979920
|
|
46 |
mT5-large |
2055.165101
|
1615.687190
|
|
29 |
T5-small |
995.334314
|
782.865991
|
|
18 |
Switch-base-8 |
425.605103
|
349.968606
|
|
16 |
T5-base |
358.614059
|
274.694340
|
|
13 |
T5-large |
253.863088
|
192.323406
|
|
Parameter |
Value |
depth |
1 |
top_k |
15 |