Metric	Score
PerplexityHashed	376.99

Metric	Score
PerplexityHashed	347.24

worst items

note: the gold standard is taken from the submission itself, not from the challenge data!

#	input	expected output	actual output	test-A PerplexityHashed
1	37bd83fd972bddf7154ab99bd328cbd0 RICHMOND ENQUIRER ChronAm 1821.2041095573313 37.538509 -77.43428 In colut idvnce with this course of proced…	ot	of:0.7121 to:0.0511 :0.1106 for:0.0277 with:0.0257 and:0.0178 in:0.0174 on:0.0134 from:0.0121 that:0.0121	9258.59
2	1424f44f79ece375ee5a11b9811b234b DEMOCRATIC MESSENGER ChronAm 1908.8510928645517 38.177063 -75.392696 A reader <>f these notes living near\n…	w'lll	to:0.5731 will:0.1130 should:0.0609 :0.1433 and:0.0269 would:0.0193 shall:0.0176 must:0.0158 can:0.0156 could:0.0144	7145.13
3	04ce38b5c98ba8fd87d85459089bba02 COURIER DEMOCRAT ChronAm 1913.2342465436327 48.76059 -98.367824 record m the office of the Register of Deed…	Per	per:0.8066 :0.1462 nper:0.0162 a:0.0131 to:0.0048 the:0.0036 and:0.0031 or:0.0025 of:0.0022 one:0.0018	7004.80
4	11a618922a619ac74b505d8e1e13f9c2 COURIER DEMOCRAT ChronAm 1911.5876712011668 48.76059 -98.367824 Order Hearing Petition for Sale of Land\nUp…	to.	to:0.7174 :0.1800 the:0.0245 in:0.0162 a:0.0122 of:0.0113 with:0.0099 by:0.0097 and:0.0095 that:0.0092	5688.32
5	494648073f3e8d8b6a2581ae7d35e668 RICHMOND ENQUIRER ChronAm 1842.9493150367834 37.538509 -77.43428 Petitions were presented and referred:.By …	lolhu	to:0.7174 :0.1800 the:0.0245 in:0.0162 a:0.0122 of:0.0113 with:0.0099 by:0.0097 and:0.0095 that:0.0092	5688.32
6	83457b2711a4c2d1d066bcd3994baa77 RICHMOND ENQUIRER ChronAm 1817.0178081874683 37.538509 -77.43428 The Senate resumed Hie consideration\nol M…	fo	to:0.7484 :0.1958 a:0.0105 and:0.0093 by:0.0079 an:0.0069 for:0.0065 that:0.0054 the:0.0048 up:0.0044	5229.30
7	3f211a6d0f9a8559013a194020247d38 RICHMOND ENQUIRER ChronAm 1831.382191749112 37.538509 -77.43428 ments of the Administration w ill, of cours…	laid,	did:0.1908 could:0.1256 would:0.0964 had:0.0792 was:0.0768 does:0.0588 is:0.0567 will:0.0527 has:0.0518 :0.2112	4848.48
8	6c486cb03e0cc9a9a5c6b406d9e63921 RICHMOND ENQUIRER ChronAm 1850.2205479134957 37.538509 -77.43428 7. He it further matted. That the person? …	other	the:0.6119 :0.2126 a:0.0509 his:0.0376 tho:0.0229 their:0.0213 an:0.0151 our:0.0093 her:0.0092 its:0.0092	4816.56
9	de516524b9cd61ab6e4f62d27b51f2a6 RAFTSMAN'S JOURNAL ChronAm 1868.5177595312184 41.02728 -78.439188 The only question left was, When shall\nm…	my	the:0.6555 :0.2241 his:0.0258 tho:0.0255 a:0.0181 their:0.0168 its:0.0088 an:0.0087 tbe:0.0085 her:0.0083	4569.85
10	1caeb07fd00e6a041bd30aebfb71abb6 RICHMOND ENQUIRER ChronAm 1857.023287639523 37.538509 -77.43428 by the Treaty of Utrecht. In 1806 the rule …	vl.lch	to:0.6097 :0.2313 will:0.0642 should:0.0191 can:0.0157 may:0.0136 shall:0.0134 and:0.0120 must:0.0106 would:0.0105	4427.59
11	d37e80ec80f3c992b2b3c6fd01b994c6 RICHMOND ENQUIRER ChronAm 1821.5794520230847 37.538509 -77.43428 those previously received. The city hud be…	beep	been:0.5995 :0.2341 had:0.0534 not:0.0267 ever:0.0211 be:0.0185 never:0.0158 always:0.0122 already:0.0096 was:0.0091	4374.20
12	596830d21161dc3dc3cdeff1ab51299f RICHMOND ENQUIRER ChronAm 1839.9602739408929 37.538509 -77.43428 hinds and Mi I in Chesterfield for V>.\nY …	first	the:0.4326 a:0.1395 :0.2365 this:0.0609 his:0.0419 their:0.0273 tho:0.0218 our:0.0157 its:0.0126 said:0.0113	4330.24
13	21a4004987635d163f83641f4f115d48 THE DICKINSON PRESS ChronAm 1897.905479420345 46.879176 -102.789624 Ill Groceries\n» We carry a full and co…	bs	be:0.6344 :0.2373 have:0.0596 bo:0.0295 he:0.0097 do:0.0090 get:0.0062 the:0.0053 take:0.0046 make:0.0043	4314.78
14	1eaf6eb3e8f56d778dac66eeb9ae2519 THE MEMPHIS APPEAL ChronAm 1886.3794520230847 35.149022 -90.051628 excited and in a hurry. This was not\nso…	had	the:0.5855 :0.2397 his:0.0485 a:0.0469 tho:0.0162 their:0.0156 its:0.0132 no:0.0128 tbe:0.0116 our:0.0100	4272.01
15	07e5d2cda2770871c9cf24e2eb3d587e DEMOCRATIC MESSENGER ChronAm 1905.382191749112 38.177063 -75.392696 they placed their canoes in the river.\…	their	of:0.6488 :0.2402 and:0.0285 in:0.0208 the:0.0176 to:0.0155 for:0.0094 ot:0.0071 or:0.0071 with:0.0050	4263.11
16	13ece120097d1dc7fca17989531a1eb6 THE INTERIOR JOURNAL ChronAm 1910.5082191463723 37.53119 -84.661888 that In local option territory alcohol\…	In	of:0.5367 :0.2422 to:0.0619 in:0.0502 for:0.0320 and:0.0228 that:0.0167 at:0.0142 by:0.0118 with:0.0115	4227.91
17	9402e6478e721417e75943b9ae6c668b RICHMOND ENQUIRER ChronAm 1833.9109588723998 37.538509 -77.43428 Gentlemen—I send you a copy of Col. Troup’…	ol	of:0.6080 :0.2502 in:0.0378 to:0.0219 that:0.0163 at:0.0148 for:0.0148 and:0.0140 on:0.0138 as:0.0084	4092.73
18	5d44cd1c02cf21930c0c7cc2b0881194 RICHMOND ENQUIRER ChronAm 1840.5532786569015 37.538509 -77.43428 Mr Rive^ emptied tin* phials of his wrath …	behind	of:0.4065 to:0.1815 :0.2502 in:0.0456 and:0.0314 for:0.0266 with:0.0170 on:0.0155 from:0.0143 by:0.0114	4092.73
19	25dea8c35ddfa304c8c5592bb882ccc3 RICHMOND ENQUIRER ChronAm 1855.4260273655505 37.538509 -77.43428 vor.common , however, only in free society…	exciting	the:0.5252 :0.2561 a:0.0906 its:0.0257 this:0.0228 his:0.0197 their:0.0192 tho:0.0191 our:0.0133 tbe:0.0082	3998.04
20	447ca324b991e9f22787e13a31980574 THE STAR OF THE NORTH ChronAm 1856.236338766191 41.004121 -76.453816 The feci is, we have in the United Sta…	this	a:0.2871 the:0.2859 :0.2562 their:0.0505 his:0.0349 an:0.0199 our:0.0196 tho:0.0161 to:0.0158 its:0.0140	3996.88
21	047d34a0127bbb1520f3999ecd202d11 RICHMOND ENQUIRER ChronAm 1824.4631147224752 37.538509 -77.43428 TUE tract of liiml on which die subscriber…	In	of:0.5560 :0.2567 in:0.0496 on:0.0309 and:0.0282 to:0.0274 for:0.0157 from:0.0153 ot:0.0114 at:0.0088	3989.09
22	c452edee0474e77184e1f077f6c1bdf8 RICHMOND ENQUIRER ChronAm 1832.6079234656447 37.538509 -77.43428 Huron is by no means certain, that in this…	<>l	of:0.4437 :0.2590 in:0.0752 by:0.0407 to:0.0404 on:0.0378 with:0.0305 from:0.0282 for:0.0267 and:0.0179	3954.06
23	69163545870f389e866c4b26a5841fbb RICHMOND ENQUIRER ChronAm 1842.0205479134956 37.538509 -77.43428 Mr. SIMMONS of Rhode I aland said lie did …	could	to:0.2375 will:0.2267 :0.2621 shall:0.0661 would:0.0561 should:0.0489 and:0.0266 must:0.0262 may:0.0261 can:0.0238	3907.30
24	969c7ff24078d6079df816ce142f2b50 RICHMOND ENQUIRER ChronAm 1862.5931506532218 37.538509 -77.43428 It. Whereas, by a General Order dated th\n…	any	the:0.6160 :0.2700 a:0.0251 tho:0.0228 his:0.0172 our:0.0154 said:0.0109 their:0.0083 this:0.0071 an:0.0070	3791.83
25	a630710b4f0d3d65481bebd6757fab91 DEMOCRAT AND SENTINEL ChronAm 1863.6506848997972 40.485194 -78.724957 is immediately cured by their applica…	on	a:0.2694 the:0.2464 his:0.1069 :0.2710 their:0.0364 its:0.0207 to:0.0136 an:0.0134 our:0.0116 tho:0.0107	3778.98
26	815f621c8cead6bd13f6a6cfd27ed667 RICHMOND ENQUIRER ChronAm 1830.5383561326737 37.538509 -77.43428 his despotism is supposed to exist. The id…	such	the:0.5214 :0.2722 a:0.0637 this:0.0496 tho:0.0256 an:0.0182 his:0.0148 our:0.0135 their:0.0114 its:0.0097	3762.32
27	04230031d64ccc9a3216e0fd38b6ddb1 WATERBURY EVENING DEMOCRAT ChronAm 1898.9794520230848 41.558153 -73.051497 is Legion, or, rather, was, for …	often,	will:0.2026 could:0.1420 can:0.1147 would:0.0695 should:0.0571 may:0.0555 :0.2747 might:0.0328 are:0.0264 must:0.0247	3727.70
28	b4188d0ad7ed014e24e185d57c9a8ed2 RICHMOND ENQUIRER ChronAm 1835.2260273655504 37.538509 -77.43428 It was found, by observation, that the wor…	nil	the:0.6022 :0.2769 a:0.0502 tho:0.0226 his:0.0134 their:0.0093 no:0.0084 tbe:0.0076 any:0.0049 an:0.0045	3698.09
29	c83d4ad9b7dc44819a0abe4ff7487b63 RICHMOND ENQUIRER ChronAm 1818.4753424340436 37.538509 -77.43428 tie it enacted by the Senate and House of …	ol	of:0.4059 :0.2787 in:0.0763 to:0.0541 and:0.0404 for:0.0384 at:0.0319 from:0.0275 on:0.0246 by:0.0221	3673.83
30	31ce674e7cdf27a3d1092c13a5066edd RICHMOND ENQUIRER ChronAm 1856.4221311159179 37.538509 -77.43428 our publi'- si" unties, anil reiiwve Virgi…	into	to:0.4671 the:0.1250 :0.2792 a:0.0243 that:0.0229 their:0.0224 and:0.0199 his:0.0152 her:0.0137 tho:0.0101	3666.89
31	0cfb5f59c681f0ab1ef48d4d9b243125 THE JASPER WEEKLY COURIER ChronAm 1894.2616438039067 38.391448 -86.930874 Moravia street Donnelly himself d…	across	of:0.4658 :0.2803 in:0.0499 to:0.0439 on:0.0383 from:0.0264 by:0.0256 for:0.0251 and:0.0231 that:0.0216	3653.23
32	2183552b0a0f5d265dd6595ce14938d2 RICHMOND ENQUIRER ChronAm 1845.4863013381532 37.538509 -77.43428 It is very well watered, there being at le…	large	of:0.5249 :0.2846 and:0.0511 to:0.0270 the:0.0257 ot:0.0235 for:0.0212 with:0.0186 or:0.0119 is:0.0116	3598.39
33	dff52282b7dfb824085420f14859f833 RAFTSMAN'S JOURNAL ChronAm 1864.2636611705627 41.02728 -78.439188 And xchereas, The amnesty therein pro-\nv…	voluntarily	be:0.2310 have:0.2171 not:0.1757 :0.2852 bo:0.0249 he:0.0194 we:0.0146 to:0.0136 never:0.0108 had:0.0079	3591.18
34	30ed5ed0482b2de4f8c6045df20b4d91 THE GRANGE ADVANCE ChronAm 1877.505479420345 44.562468 -92.533801 awake, standing uponWe'seatsVitn'^tlie\nc…	His	the:0.4901 :0.2947 a:0.0834 an:0.0304 tho:0.0240 his:0.0193 their:0.0171 in:0.0145 by:0.0141 and:0.0124	3474.72
35	0819f2144e9543f9ddd6b15004c6b3e5 LOS ANGELES HERALD ChronAm 1909.2041095573313 34.054935 -118.244476 These tacts point conclusively, in my J…	Under	of:0.3261 in:0.1644 :0.2954 to:0.0578 that:0.0453 for:0.0365 and:0.0275 on:0.0225 from:0.0133 with:0.0114	3467.18
36	54b2d81f2511430927f75e42e8e11f02 RICHMOND ENQUIRER ChronAm 1833.7958903792492 37.538509 -77.43428 Ihe liraneh RKhUSKI) lo receive on deposit…	he	the:0.6304 :0.2962 tho:0.0207 a:0.0160 of:0.0090 his:0.0074 tbe:0.0066 and:0.0062 an:0.0040 this:0.0035	3457.12
37	19e0b8e943e282b9a84a281ec17d59e7 THE REVIEW ChronAm 1921.4945205162355 35.955692 -80.005318 To make certain that I avoided any\npossibility …	ample,	the:0.3404 this:0.1502 :0.2963 a:0.0519 any:0.0353 some:0.0325 every:0.0269 that:0.0256 their:0.0221 which:0.0188	3455.96
38	ae2ae63e8b50111dc91a0d3dd0790c1e RICHMOND ENQUIRER ChronAm 1844.3948087115461 37.538509 -77.43428 The reasons ass gned in 1S25, 1827, 1>29, …	by	of:0.5229 :0.2983 in:0.0473 and:0.0288 with:0.0247 that:0.0226 to:0.0212 on:0.0168 for:0.0093 from:0.0082	3433.13
39	2f3bc7e5b5485966c2b817c24b2c0545 GREAT FALLS DAILY TRIBUNE ChronAm 1920.3292349410544 47.504885 -111.291891 In Ohio Senator Harding, the fav…	are	the:0.3648 a:0.2536 :0.3065 an:0.0177 tho:0.0121 to:0.0106 he:0.0102 his:0.0088 their:0.0079 it:0.0078	3340.95
40	d7373db9f384b82b854c48406dd5f73d RICHMOND ENQUIRER ChronAm 1815.7849314751395 37.538509 -77.43428 teract. On the conti ary, those principles…	an	the:0.3564 a:0.2086 :0.3068 his:0.0533 its:0.0161 their:0.0160 said:0.0145 tho:0.0137 our:0.0076 this:0.0070	3337.68