9bde718de8c71fd9e7c7671cae39dfc09faad7bf
Transformer encoder-decoder models (seq2seq) transformer huggingface-transformers transformer-encoder-decoder seq2seq

 

challenge
GLUE-LM-GAP
submitter
kaczla
submitted
2023-03-06 15:48:25.968755 UTC
# model_name dev-0 PerplexityHashed test-A PerplexityHashed
152 LongT5-Local-base 1006246.726444 1008375.972086
130 ByT5-base 854668.733905 855967.364657
125 ByT5-small 577506.468555 584584.351424
124 FLAN-T5-large 544241.485156 562529.862349
123 T5-efficient-mini 493098.539241 510519.379968
117 T5-efficient-tiny 408282.416232 418979.060903
116 T5-large-v1_1 420124.898821 415089.099066
112 FLAN-T5-small 306911.104213 324604.408047
109 T5-large-v1_1-lm-adapt 337413.972666 300663.100084
108 T5-small-v1_1 290626.365308 282303.546243
107 T5-efficient-small 303742.494167 277887.449243
106 T5-base-v1_1 290005.373172 267972.646440
102 T5-small-v1_1-lm-adapt 239393.520998 233990.717024
101 LongT5-TGlobal-base 199552.610743 211414.917005
96 T5-efficient-base 152750.729495 124430.988625
95 FLAN-T5-base 103892.908235 118104.568290
94 T5-efficient-large 105072.298767 84932.099781
92 T5-base-v1_1-lm-adapt 74652.507386 63737.130182
61 mT5-small 4691.781349 3976.321841
55 mT5-base 3046.209545 2491.979920
46 mT5-large 2055.165101 1615.687190
29 T5-small 995.334314 782.865991
18 Switch-base-8 425.605103 349.968606
16 T5-base 358.614059 274.694340
13 T5-large 253.863088 192.323406
Parameter Value
depth 1
top_k 15