5b1faa715a9cb141a2c65edc034180803279d61d
Training only classifier of pretrained resnet-50 neural-network cnn faster-r-cnn

 

challenge
The task of image retrieval in historical publications
submitter
[anonymized]
submitted
2022-03-15 00:55:15.750257 UTC
file basename
out
Parameter Value
batch_size 16
epochs 20
learning-rate '3e-4'
lr_gamma 0.4
lr_scheduler true
lr_step_size 5
num_workers 2
rescale ! '[1000, 1000]'
trainable_backbone_layers 5
weight_decay 0

test-A / 909200b1b8c4ce28d0d01617d8a7618fbeeafa05
Metric Score
F1 71.24
P 62.59
R 82.65
F1 P R
photograph 66.43 56.26 81.10
map 16.81 9.37 81.31
cartoon 49.21 35.67 79.29
headline 76.19 70.33 83.13
editorial_cartoon 29.28 18.09 76.72
illustration 26.77 16.56 69.85
advertisement 76.58 70.78 83.41

dev-0 / 6a8dfdcef97931209adedba3abbb0e2cd0cc7169
Metric Score
F1 70.14
P 61.14
R 82.25
F1 P R
photograph 63.77 52.35 81.58
map 19.82 11.16 88.30
cartoon 49.28 37.25 72.78
headline 76.77 70.95 83.63
editorial_cartoon 28.18 19.03 54.28
illustration 25.49 15.31 76.03
advertisement 73.80 67.60 81.24

worst items

note: the gold standard is taken from the submission itself, not from the challenge data!
# input expected output actual output test-A F1
1 3384.jpg editorial_cartoon:210,414,4981,6530 advertisement:25,414,4925,6446 map:165,916,3540,6352 photograph:5,494,2537,5403 illustration:0,354,4935,6432 advertisement:1813,381,4940,476… 0.00
2 1315.jpg photograph:108,648,5130,2760 photograph:182,599,3384,2600 photograph:3084,671,5181,2811 photograph:144,2422,2606,4931 photograph:112,4608,1861,6425 photograph:2429,2574,… 10.42
3 2148.jpg illustration:2238,132,3414,432 advertisement:1020,504,1734,5354 advertisement:1752,504,2472,6368 advertisement:3235,4153,3955,6337 advertise… advertisement:1020,1656,1741,2000 advertisement:3945,2900,4677,3512 headline:276,5470,1014,5745 advertisement:1730,494,2468,1113 advertiseme… 15.31
4 2287.jpg photograph:3966,5407,5106,6829 photograph:528,3679,1650,5101 photograph:4002,2076,5142,3516 photograph:504,324,1644,1776 advertisement:354,1… advertisement:310,102,5420,6983 headline:457,5129,5312,5430 advertisement:135,4873,5341,6983 headline:237,1780,5437,2110 headline:593,439,53… 15.90
5 2379.jpg photograph:3266,234,5361,2280 photograph:78,270,3200,1932 photograph:86,5648,2534,8101 photograph:3821,5632,5395,8011 photograph:3202,237,5270,3065 photograph:92,245,3175,2329 photograph:2404,5330,3… 19.57
6 3527.jpg map:174,3337,2550,6440 cartoon:156,360,4938,3331 illustration:2556,3343,4932,6500 map:215,3062,2665,6429 map:2285,3422,4951,6436 advertisement:2573,3380,5002,6311 headline:421,346,4730,561 illustration:2619,3346,4899,6574 … 22.94
7 1216.jpg photograph:126,366,5331,6362 photograph:180,366,5259,4147 photograph:210,420,5241,3427 photograph:207,471,3204,2737 photograph:2410,3281,5220,6313 photograph:213,4341,2729,6345 photograph:21,2501,2548,4558 photograph:3241,530,5… 23.46
8 2619.jpg cartoon:3588,1134,5316,3661 headline:3583,7684,4471,8024 headline:3564,7702,4477,8400 advertisement:3576,7684,4458,8535 cartoon:3376,1209,5365,3707 map:3129,1101,5592,3… 23.75
9 1095.jpg illustration:2059,2316,2911,4218 headline:246,678,1104,1584 headline:2935,5160,3799,5448 headline:2971,684,3817,1290 advertisement:3812,642,… headline:235,580,1107,1642 advertisement:3831,598,4741,1125 advertisement:4722,3601,5644,4400 advertisement:4741,3121,5638,3620 advertisemen… 23.87
10 2474.jpg photograph:48,174,1920,3402 advertisement:3079,132,5461,4789 advertisement:2316,4777,5533,7927 advertisement:3101,199,5498,5000 advertisement:3939,6795,5498,7784 photograph:27,2743,1402,5367 advertisement:2335,5351,3928,6994 advertisem… 23.99
11 187.jpg editorial_cartoon:3073,186,5179,2857 editorial_cartoon:3067,180,5197,2857 headline:3097,2887,4477,3145 advertisement:936,198,1608,456 advert… headline:901,158,1627,436 headline:3089,2866,4487,3144 headline:4514,2873,5213,3039 headline:186,3151,896,3290 editorial_cartoon:3100,105,52… 24.62
12 147.jpg map:4437,1104,6574,2628 illustration:5487,318,6561,912 advertisement:4437,6210,6634,10290 advertisement:84,360,2317,3408 advertisement:2311,… advertisement:178,325,2312,3368 headline:116,3463,1228,4103 advertisement:4426,6191,6602,10137 advertisement:2285,346,3362,4449 headline:335… 29.52
13 995.jpg photograph:276,2658,960,3468 advertisement:264,4267,1740,6661 advertisement:4670,3859,6152,6487 advertisement:4670,3216,6152,3870 advertisem… advertisement:4672,3202,6118,3837 advertisement:4647,2263,6149,3188 headline:1711,5204,2459,5445 advertisement:272,4272,1711,6646 advertisem… 29.99
14 840.jpg cartoon:3151,234,4933,2593 advertisement:1926,3793,3156,6212 advertisement:1920,1584,2520,1878 advertisement:1926,318,2520,576 advertisement… cartoon:3152,4000,3748,4830 advertisement:1879,3815,3107,6221 cartoon:3728,2648,4976,4141 cartoon:4355,4179,4981,5481 advertisement:1313,490… 30.33
15 1988.jpg photograph:222,1026,6619,4045 photograph:1260,3751,3396,8595 photograph:902,3781,3424,8579 illustration:0,856,6507,4082 photograph:157,714,6596,4157 illustration:485,3499,3479,8579 illustration:54,733,… 31.01
16 2402.jpg illustration:306,8605,6727,9187 photograph:132,264,7087,4776 headline:186,384,6769,774 headline:233,439,6974,821 photograph:70,401,6854,5054 illustration:49,745,3685,4719 photograph:3735,592,6861,4834 photograph:113,831,3373,46… 36.23
17 1079.jpg photograph:3704,2520,4850,3390 illustration:2996,3966,3686,4878 illustration:966,816,1644,1920 headline:270,396,1657,840 headline:2353,402,3… advertisement:2970,5570,3654,6077 advertisement:2986,4868,3659,5550 headline:251,394,1655,854 headline:3726,380,5073,614 headline:2333,394,3… 36.80
18 2836.jpg cartoon:348,180,6619,9019 cartoon:386,157,6652,9051 cartoon:303,8439,6908,8968 map:359,4911,2625,6610 illustration:2618,269,6479,1188 39.31
19 577.jpg photograph:1518,2130,2196,3414 headline:144,366,852,1770 headline:1506,354,2166,840 headline:1530,4993,2208,5197 headline:139,343,849,1767 headline:1502,343,2180,854 headline:854,2551,1522,2758 photograph:1522,2104,2191,3380 headline:3527,4028,4185,4189… 39.40
20 3311.jpg editorial_cartoon:3662,5096,4989,6554 editorial_cartoon:2239,618,3884,2682 editorial_cartoon:2245,5108,3644,6530 cartoon:1549,4604,2221,6080… cartoon:150,2886,885,3941 cartoon:1559,4629,2239,6099 cartoon:905,2892,2214,4549 cartoon:166,3968,870,4970 cartoon:191,661,2224,2839 cartoon… 39.99
21 412.jpg cartoon:4111,2724,4789,4266 cartoon:1932,456,3396,1446 advertisement:3367,654,4111,1008 advertisement:4117,666,4831,1008 advertisement:3367,… advertisement:3351,4267,4841,6834 advertisement:3372,3091,4109,4238 headline:2640,5307,3377,5536 advertisement:3367,659,4861,3880 advertisem… 40.12
22 780.jpg cartoon:294,8580,6447,9258 cartoon:294,354,6519,4494 cartoon:294,4524,6435,8496 cartoon:482,4405,6437,8661 cartoon:206,289,6512,4555 cartoon:158,8530,6547,9064 editorial_cartoon:323,6650,2281,8586 editorial_cartoon:1895,… 41.82
23 1629.jpg photograph:276,4448,5349,6393 photograph:204,468,5331,4448 photograph:207,505,5397,4618 photograph:321,3974,5452,6400 illustration:239,564,5250,4532 photograph:228,3077,1690,4299 photograph:136,3223,… 42.95
24 452.jpg cartoon:3114,5022,4302,6384 cartoon:4320,4812,4950,6396 editorial_cartoon:3066,2484,4326,3384 cartoon:3066,3390,3678,3984 cartoon:3078,216,4… cartoon:3028,238,4974,2384 advertisement:2456,2861,3080,4318 advertisement:608,311,1795,914 cartoon:3096,5054,4313,6345 advertisement:1852,4… 44.49
25 652.jpg photograph:546,552,2940,4087 advertisement:4489,2136,6097,6427 advertisement:4495,6416,6097,8276 advertisement:600,6134,2154,7268 advertisem… advertisement:577,6140,2141,7251 advertisement:4476,6380,6111,8190 advertisement:590,7276,2166,8267 advertisement:5248,1323,6046,1674 advert… 44.92
26 973.jpg illustration:258,8382,6378,8904 illustration:162,2808,2070,4356 illustration:4188,1734,6432,5328 photograph:2064,1128,4578,5340 illustration… illustration:215,694,6711,5437 headline:2134,5419,6395,5711 illustration:100,2723,2093,4386 illustration:188,8407,6287,8873 map:4113,1050,64… 45.22
27 182.jpg editorial_cartoon:2694,186,4836,2526 editorial_cartoon:2718,222,4812,2478 headline:2748,2586,4104,2808 headline:1374,3967,2034,4129 headline… headline:2720,2580,4088,2786 headline:609,212,1318,406 headline:1343,3998,2022,4104 editorial_cartoon:2735,199,4782,2830 headline:2036,231,2… 45.99
28 934.jpg photograph:2900,3114,3602,4992 headline:210,804,912,2226 headline:1543,792,2227,2220 headline:2876,774,3560,2202 headline:918,2940,1560,3150… headline:2849,759,3551,2227 headline:1523,766,2225,2253 advertisement:4243,4707,4898,5064 headline:197,792,910,2259 headline:904,2941,1560,3… 46.39
29 572.jpg cartoon:3678,4040,4944,5108 cartoon:3060,4316,3666,5096 cartoon:4272,2671,4950,4045 cartoon:3030,294,4920,2629 cartoon:3114,5115,4890,6423 c… advertisement:587,5243,1808,6506 advertisement:1823,4532,3029,6473 cartoon:3060,2671,4328,4014 advertisement:2442,3721,3060,3941 cartoon:306… 48.28
30 2365.jpg illustration:2569,1608,3307,2904 illustration:1110,2665,1836,3961 headline:402,366,1152,1422 advertisement:4009,4165,4747,4717 advertisement… advertisement:4001,4178,4728,4737 illustration:1113,2687,1835,4027 headline:432,394,1093,1340 headline:4006,3110,4733,3217 illustration:2557… 49.15
31 1128.jpg map:4508,2118,6309,4344 headline:4520,2106,6303,2478 advertisement:60,8377,6327,8845 headline:5403,6007,6291,6217 headline:5427,7111,6315,72… cartoon:19,457,6250,2150 map:4498,2214,6317,4228 advertisement:33,8364,6384,8795 headline:4531,2104,6324,2480 illustration:4538,5225,6324,58… 49.59
32 1218.jpg photograph:1993,378,3818,2995 headline:312,438,2017,900 headline:300,4268,2017,4760 headline:300,6177,2023,6657 headline:3759,828,5524,960 h… headline:302,4273,2030,4731 headline:3770,2835,5493,2993 headline:308,6178,2042,6643 headline:3741,4373,5487,4581 headline:3759,6178,5522,63… 50.06
33 1420.jpg illustration:4178,4039,5102,5683 photograph:1404,1464,2292,4171 advertisement:5084,960,6842,2316 advertisement:5102,2292,6830,2670 advertise… headline:478,983,2364,1441 advertisement:5069,1097,6940,6928 advertisement:5090,7387,6925,9143 headline:3227,3063,4155,3454 headline:3227,54… 50.52
34 729.jpg photograph:1506,324,2160,1272 headline:2136,324,2772,858 advertisement:906,378,1506,1602 headline:942,3463,1524,3547 headline:924,4537,1506,… headline:2156,315,2785,887 advertisement:902,392,1518,1589 photograph:1513,333,2160,1268 advertisement:3427,4161,4061,4656 headline:2785,315… 50.86
35 1179.jpg illustration:2016,2760,3222,4656 photograph:3799,402,4417,1560 headline:168,420,1374,852 headline:1374,414,3210,606 headline:1374,606,1992,1… headline:126,407,1378,865 advertisement:2019,4688,3230,6279 advertisement:3225,3390,5002,6253 headline:1383,2423,1993,2837 headline:1373,407… 51.82
36 32.jpg cartoon:1321,300,3368,2413 advertisement:3362,4227,4701,6484 advertisement:3362,3740,4028,4022 advertisement:4053,3548,4689,4232 advertisement:3347,4235,4737,6559 advertisement:25,3931,647,4498 cartoon:1319,324,3372,2431 advertisement:3362,3728,4019,4120 headline:2679,… 52.39
37 3488.jpg map:372,426,2563,2520 headline:330,3888,1080,4536 advertisement:3326,4524,4797,5370 advertisement:3326,5382,4785,6222 advertisement:3290,625… headline:355,3882,1087,4540 advertisement:3297,4518,4822,5328 advertisement:3307,3593,4049,4460 advertisement:2555,3578,3302,6926 advertisem… 52.91
38 1144.jpg cartoon:252,5175,942,6321 cartoon:954,336,1620,1470 advertisement:276,330,936,2353 headline:948,1470,1608,1950 headline:1620,330,2286,558 he… headline:267,342,953,867 headline:930,1459,1611,1963 advertisement:3609,5278,4309,5910 advertisement:2946,2171,3591,3543 advertisement:2960,… 54.28
39 691.jpg editorial_cartoon:3038,300,5013,3054 headline:3110,3084,4461,3246 headline:4569,3078,5181,3210 headline:864,300,1566,504 headline:848,295,1583,517 headline:3058,3060,4506,3241 editorial_cartoon:3053,262,5220,3067 headline:4533,3060,5247,3201 headline:124,4540,8… 54.33
40 2108.jpg cartoon:2226,270,4998,2622 cartoon:4339,4410,4987,7032 headline:2928,2634,4314,2826 cartoon:2214,310,5020,2605 headline:2929,2620,4281,2834 headline:135,5469,832,5587 cartoon:4323,4384,5008,7063 headline:2923,3904,3590,4015 … 55.36

Compare with other submission