Transformer decoder models (CLM) transformer huggingface-transformers transformer-decoder clm
- challenge
- GLUE-LM-GAP
- submitter
- kaczla
- submitted
- 2023-02-13 09:07:21.810036 UTC
- original repo
https://gitlab.com/kaczla/glue-lm-gap.git / branch decoder
- publicly available at
git://gonito.net/glue-lm-gap / branch submission-07997- browsable at
- https://gonito.net/gitlist/glue-lm-gap.git/submission-07997/
- clone by
git clone --single-branch git://gonito.net/glue-lm-gap -b submission-07997
| # |
model_name |
dev-0 PerplexityHashed |
test-A PerplexityHashed |
|
|---|
| 93 |
PolishGPT-2-small |
79627.908898
|
79278.556763
|
|
| 88 |
PolishGPT-2-medium |
59754.664598
|
58446.342067
|
|
| 85 |
PolishGPT-2-large |
50521.897871
|
47767.398812
|
|
| 84 |
GPT-fr-base |
46956.696551
|
46362.459344
|
|
| 83 |
GPT-fr-small |
38383.860649
|
38199.695407
|
|
| 72 |
DistilGPT-2 |
8885.673813
|
9714.724445
|
|
| 70 |
BioGPT |
10345.246977
|
9377.074086
|
|
| 68 |
Pythia-70M |
7424.531362
|
7329.966876
|
|
| 67 |
Pythia-160M |
5019.867530
|
4845.613045
|
|
| 66 |
GPT-2-base |
4732.154178
|
4700.318867
|
|
| 62 |
GPT-Neo-125M |
4711.535675
|
4373.950153
|
|
| 60 |
Pythia-410M |
3894.413763
|
3685.007192
|
|
| 54 |
GPT-2-medium |
2591.721115
|
2250.943823
|
|
| 51 |
OPT-125M |
2560.284962
|
2021.615293
|
|
| 49 |
OPT-350M |
2257.973076
|
1767.228754
|
|
| Parameter |
Value |
|---|
| depth |
1 |
| top_k |
15 |