Metric	Score
PerplexityHashed	454.57

Metric	Score
PerplexityHashed	411.07

worst items

note: the gold standard is taken from the submission itself, not from the challenge data!

#	input	expected output	actual output	test-A PerplexityHashed
1	a46a5a890b809b63b0644f1f3f50d16e RICHMOND ENQUIRER ChronAm 1822.17397257103 37.538509 -77.43428 »v hat officers nave been specially appointe…	tion,	tion:0.8556881546974182 tions:0.08477835357189178 tive:0.007084242533892393 tional:0.0046079764142632484 tary:0.003017581067979336 tlon:0.00…	24169.69
2	2c32368440ed329f43c57467551285aa RICHMOND ENQUIRER ChronAm 1848.7445354875026 37.538509 -77.43428 limns,replete with argument, wit and anec\…	ler	ter:0.6300528645515442 <unk>:0.24531909823417664 ters:0.040888458490371704 a:0.01333839725703001 the:0.008594038896262646 and:0.003310089930…	19375.14
3	431bc42c9bae960d7ba5324262ea6f4f THE REVIEW ChronAm 1917.1821917491122 35.955692 -80.005318 "I stand here and nave taKen me\nhigh and solemn…	sities	sary:0.7662391066551208 sity:0.09072024375200272 <unk>:0.04802193492650986 sarily:0.028019573539495468 to:0.006286866497248411 is:0.00373269…	19172.68
4	0b4f47c072781044dc785153b75ff5b4 THE WYANDOT PIONEER ChronAm 1860.872950788049 40.827279 -83.281309 We are enabled here to offer the communi…	tion.	tion:0.7016712427139282 tional:0.17960698902606964 ents:0.024875549599528313 tions:0.02179657481610775 ted:0.00448148837313056 <unk>:0.00362…	17030.78
5	483b31c888630a6712df48fe6cee758d RICHMOND ENQUIRER ChronAm 1831.2397259956874 37.538509 -77.43428 11. The said superior courts shall have th…	y	try:0.44460901618003845 ty:0.20725949108600616 cil:0.10136263072490692 ties:0.08923399448394775 <unk>:0.05545205622911453 tries:0.0272968132…	16003.90
6	d9284676477f5d3c6cf470e2aba86f6a RICHMOND ENQUIRER ChronAm 1818.1547944888382 37.538509 -77.43428 It has hcon admitted hy the exccnli e\nCou…	cil,	try:0.44460901618003845 ty:0.20725949108600616 cil:0.10136263072490692 ties:0.08923399448394775 <unk>:0.05545205622911453 tries:0.0272968132…	16003.90
7	1f61d0be115721333a36e20b0f91f639 THE MINERS ChronAm 1852.3811475093605 42.500622 -90.664797 The people of L.wa demanded a new set of\nlaws, …	od,	ods:0.8076410889625549 <unk>:0.06569260358810425 od:0.034964557737112045 the:0.005327056627720594 of:0.0038369514513760805 «:0.0034012540709…	13805.92
8	2b4578faaa5d040beefbcf8ece106e6e RICHMOND ENQUIRER ChronAm 1816.8483606241145 37.538509 -77.43428 The funds opened and shut todayatth\nsame …	tent	tem:0.8044132590293884 <unk>:0.051604919135570526 tems:0.03224235400557518 tern:0.010553351603448391 a:0.005979849956929684 the:0.0055500906…	12688.56
9	25a2c22ec0d42967dbd94992792887b5 THE DICKINSON PRESS ChronAm 1914.8315068176053 46.879176 -102.789624 Mr. Norton made his remarks very\ninte…	ture.	ture:0.7323468327522278 tures:0.18293137848377228 and:0.0010535579640418291 <unk>:0.0008550070924684405 at:0.0006257162312977016 on:0.000547…	12673.42
10	185cbc4f717abceebbcc0cddce445938 THE MINERS ChronAm 1854.2205479134957 42.500622 -90.664797 pow d\ncerning it hs true,lie is cany\nin New Vo…	ry	dence:0.4155597984790802 dently:0.24315884709358215 dent:0.1084904745221138 dences:0.050695206969976425 denced:0.04273168370127678 <unk>:0.0…	10303.11
11	11a618922a619ac74b505d8e1e13f9c2 COURIER DEMOCRAT ChronAm 1911.5876712011668 48.76059 -98.367824 Order Hearing Petition for Sale of Land\nUp…	to.	to:0.8195013999938965 <unk>:0.023200644180178642 the:0.010426430031657219 in:0.008341914974153042 by:0.007026867941021919 for:0.006896985229…	9069.32
12	494648073f3e8d8b6a2581ae7d35e668 RICHMOND ENQUIRER ChronAm 1842.9493150367834 37.538509 -77.43428 Petitions were presented and referred:.By …	lolhu	to:0.8195013999938965 <unk>:0.023200644180178642 the:0.010426430031657219 in:0.008341914974153042 by:0.007026867941021919 for:0.006896985229…	9069.32
13	14453ce06937921547bd704890587cbf EVENING CAPITAL ChronAm 1884.8702185476118 38.97864 -76.492786 Should your soil be new, perhaps a\npiece of…	ficient.	ficient:0.2536017596721649 fering:0.24003159999847412 <unk>:0.11145348101854324 fered:0.10930424183607101 ficiently:0.0861038789153099 frage…	8161.44
14	c954306d673d88d08987736c007f2dd6 THE WYANDOT PIONEER ChronAm 1868.5396174547157 40.827279 -83.281309 warrant in the Constitution, and if\nsa…	of.	of:0.4259010851383209 to:0.3752242624759674 for:0.022149940952658653 and:0.011312015354633331 in:0.010434780269861221 ot:0.00855820812284946…	7793.35
15	06c470adf4a46c7204e72274d92deb7c THE DICKINSON PRESS ChronAm 1917.264383529934 46.879176 -102.789624 NOTICE IS HEREBY GIVEN That\nunder auth…	ter,	ter:0.579727828502655 <unk>:0.10403338074684143 ters:0.08988496661186218 tered:0.03223501518368721 rel:0.028925886377692223 terly:0.01523525…	7249.80
16	47fae90e685764536e61f8b538f18357 RICHMOND ENQUIRER ChronAm 1816.301912536683 37.538509 -77.43428 thousand m*m were necessary for Malta,\nG b…	e	chise:0.33753085136413574 cisco:0.28647592663764954 cis:0.10753394663333893 <unk>:0.10114381462335587 tically:0.013152435421943665 and:0.004…	7155.48
17	26824b6b961482d5ad7e90c8fbcb575d THE JASPER WEEKLY COURIER ChronAm 1901.3164383244546 38.391448 -86.930874 The war department lias now sole\…	istrative	<unk>:0.313225120306015 istration:0.2909891903400421 istered:0.12889695167541504 ister:0.0993134155869484 per:0.006569961551576853 year:0.00…	7042.12
18	9a23949c6410e0a789c008623c370c36 DEMOCRAT AND SENTINEL ChronAm 1866.891780790208 40.485194 -78.724957 The present President of the United\nS…	States.	states:0.6805772185325623 <unk>:0.0677398145198822 slates:0.040241632610559464 state:0.02012944407761097 and:0.012155949138104916 stales:0.0…	6794.18
19	0ba3ca99842e04d54ea7df3d3133e753 CLARKSVILLE CHRONICLE ChronAm 1861.6424657217149 36.527761 -87.35887 supposed that imprisonment for debt hu…	Mates,	states:0.6805772185325623 <unk>:0.0677398145198822 slates:0.040241632610559464 state:0.02012944407761097 and:0.012155949138104916 stales:0.0…	6794.18
20	0b10beeb0214b3ac3f65555659106adf ARIZONA REPUBLICAN ChronAm 1910.6205479134958 33.448587 -112.077346 pounds',, against ,488,000,000 In 1909,…	States'	states:0.6805772185325623 <unk>:0.0677398145198822 slates:0.040241632610559464 state:0.02012944407761097 and:0.012155949138104916 stales:0.0…	6794.18
21	104c6456c01727cf0ac8932be19be006 EVENING CAPITAL ChronAm 1884.6352458700162 38.97864 -76.492786 Professor Voelker, the agricultural\nchemist…	ment,	ment:0.6963628530502319 <unk>:0.06924592703580856 ments:0.03062267415225506 ing:0.015378337353467941 the:0.011278974823653698 a:0.0083561344…	6517.49
22	3f439f01b13bddaec7d3ee4f4feecd9f RICHMOND ENQUIRER ChronAm 1860.2937158153713 37.538509 -77.43428 1 resume iny narrative at the point \\vl\n…	abored	of:0.6659713983535767 and:0.06257770955562592 to:0.04194486886262894 for:0.018425466492772102 or:0.01667950488626957 in:0.013048229739069939…	6422.02
23	bcbcc8cb2d92e94cb04e79cb2a2c5271 RICHMOND ENQUIRER ChronAm 1834.582191749112 37.538509 -77.43428 ' V hen Mr. Calhoun’s dreaded rival, Mr Cra…	live	tive:0.5080100297927856 tion:0.2637588083744049 ted:0.017930416390299797 tions:0.016567640006542206 tors:0.008452710695564747 tives:0.007493…	6208.73
24	de111a1d125c8de6438046abae481278 RICHMOND ENQUIRER ChronAm 1825.9602739408929 37.538509 -77.43428 lo al irm the patriot”? With die opinions …	ng,	ing:0.741420567035675 <unk>:0.05348464846611023 ed:0.008871391415596008 a:0.0072152139618992805 and:0.005843821447342634 the:0.0037826350890…	5938.99
25	366e401abc352060a99543482122e40d THE WICHITA CITY EAGLE ChronAm 1879.8095890093862 37.692236 -97.337545 Sunday morning, refused permission t…	nor	of:0.7257406115531921 and:0.029781481251120567 <unk>:0.02017206884920597 ot:0.012539379298686981 for:0.00975117925554514 or:0.00863368343561…	5661.75
26	1229cf5042d1e798830034bd2b110f78 BRADFORD OPINION ChronAm 1877.9356164066464 43.994599 -72.127742 The total exports during the last fiscal y…	by	angeles:0.5290300846099854 <unk>:0.12273643165826797 augeles:0.0673493817448616 ange:0.027948882430791855 the:0.021342575550079346 a:0.02080…	5456.41
27	5293591fac43107d0c4c40f90adee918 DELAWARE REPUBLICAN ChronAm 1873.7520547628108 39.745947 -75.546589 pealed, to disgrace the Stale ; they\nt…	States	states:0.67475426197052 <unk>:0.056477244943380356 statos:0.048558034002780914 and:0.007865723222494125 state:0.007270221132785082 slates:0.…	5396.95
28	53b314958babe6fbde1dd312915114bf RICHMOND ENQUIRER ChronAm 1854.1027396943175 37.538509 -77.43428 into the expediency of appropriating S4.50…	to,	to:0.4767318069934845 upon:0.09971241652965546 that:0.09902885556221008 with:0.040401458740234375 in:0.028483998030424118 on:0.0240333955734…	5392.46
29	3ff22426092d3a6869b2bd48364ba5f2 RAFTSMAN'S JOURNAL ChronAm 1858.9356164066464 41.02728 -78.439188 aware that the earliest and the best frui…	tages.	tage:0.3917490839958191 tages:0.31914469599723816 <unk>:0.06105423346161842 and:0.0123145068064332 of:0.007233192678540945 the:0.00697245588…	5368.58
30	0d1f0da2e21be820a88a3b831f4af4ba THE JASPER WEEKLY COURIER ChronAm 1912.1270491487048 38.391448 -86.930874 of doing this has been found than…	he	to:0.6288368105888367 of:0.07291305810213089 <unk>:0.03249155357480049 and:0.01997091993689537 thereto:0.013604328967630863 ;:0.013276644982…	5206.06
31	1480d040bf88ccd74e068709857438bb GREAT FALLS DAILY TRIBUNE ChronAm 1920.8374316623658 47.504885 -111.291891 Bonds numbered 1 to 7, inclusive…	sive,	sive:0.7942522764205933 of:0.0024825981818139553 <unk>:0.0014016510685905814 ment:0.0009351128246635199 thereof:0.000735555833671242 ance:0.…	5167.13
32	ee21a1f4d082e7ebf867d3506aeba084 MOUNTAIN HOME REPUBLICAN ChronAm 1925.7712328450025 43.13295 -115.691197 H. A. Beach and John W. Caldwell,\…	sive,	sive:0.7942522764205933 of:0.0024825981818139553 <unk>:0.0014016510685905814 ment:0.0009351128246635199 thereof:0.000735555833671242 ance:0.…	5167.13
33	47760e14f5f772dc8bc8d84795614664 FREMONT WEEKLY FREEMAN ChronAm 1852.0068305694697 41.35033 -83.121863 unite with the Galena road, over whic…	nor	of:0.6514722108840942 and:0.04603082686662674 to:0.024695266038179398 <unk>:0.024236053228378296 in:0.022463670000433922 on:0.01358704082667…	5140.38
34	4d41aef961aa1d8bd1a90480f2a68d50 ARIZONA SILVER BELT ChronAm 1883.2260273655504 33.399478 -110.87095 Rfsolved, That whilst wo do not\nfor th…	That	that:0.4469597637653351 to:0.21677346527576447 by:0.05874578654766083 on:0.01658044196665287 the:0.015652811154723167 and:0.0155418319627642…	4925.43
35	86e6a88018dda3cc324889dd32bb5411 RICHMOND ENQUIRER ChronAm 1846.5027396943176 37.538509 -77.43428 By way ol Brazos Santiago we received yes¬…	Antonio	not:0.7240871787071228 <unk>:0.036352574825286865 the:0.006988092325627804 and:0.005685826763510704 in:0.0054072304628789425 a:0.00454832939…	4922.49
36	f1479cb9a6256b326867f7c18fb195e5 EVENING CAPITAL ChronAm 1923.3849314751394 38.97864 -76.492786 room dwelling. Numbered . ".5 and 37. wltl\n…	o	in:0.5117445588111877 lying:0.093587227165699 on:0.06348325312137604 and:0.04136449098587036 at:0.0391358956694603 <unk>:0.01877486892044544…	4831.79
37	4069ae74c8d2c08a5502d5556131dde4 LINCOLN COUNTY ADVOCATE ChronAm 1877.064383529934 43.300815 -96.59282 hours, must pads through oae degree,\…	Francisco	francisco:0.43326014280319214 <unk>:0.15973947942256927 antonio:0.040336884558200836 pedro:0.035432904958724976 fran:0.032386261969804764 ju…	4826.79
38	0107777cc57db3d0f0271976e60befa9 ARIZONA REPUBLICAN ChronAm 1916.1680327552622 33.448587 -112.077346 bridge within x few days, according\nto…	Francisco;	francisco:0.43326014280319214 <unk>:0.15973947942256927 antonio:0.040336884558200836 pedro:0.035432904958724976 fran:0.032386261969804764 ju…	4826.79
39	7c70ca8410681e471df25898edb1feda RICHMOND ENQUIRER ChronAm 1839.6808218860983 37.538509 -77.43428 There is an eft'ort making ("being made") …	lo	to:0.6017866730690002 of:0.04443752393126488 and:0.041777294129133224 in:0.02795008383691311 for:0.023055007681250572 the:0.0210968255996704…	4816.20
40	39c705cf07381ee0e5703e69bc1b6594 RICHMOND ENQUIRER ChronAm 1852.9030054328578 37.538509 -77.43428 The foil >wing is the message from the Pre…	guarantees	of:0.5587658286094666 and:0.08834697306156158 <unk>:0.050175245851278305 in:0.029087180271744728 to:0.02399986796081066 or:0.012071494013071…	4787.17