A CHARACTER-LEVEL TRANSFORMER TRAINED FROM SCRATCH
15,540 MATCHES THAT HAPPENED · AN INFINITE NUMBER THAT DIDN'T
Low repeats the corpus. High invents countries.
STRUCTURE WITHOUT MEANING
TRAINED FROM RANDOM INITIALISATION ON 2.5 MB OF INTERNATIONAL MATCH RECORDS 1872–2026 · CORPUS MARTJ42/INTERNATIONAL_RESULTS · 15,540 FIXTURES WITH GOALSCORERS · CHARACTER-LEVEL TOKENISER, 197 TOKENS · ARCHITECTURE 6 LAYERS, 8 ATTENTION HEADS, 256 EMBEDDING DIMENSIONS, 256 CHARACTER CONTEXT · 4,901,061 PARAMETERS · TRAINED 5,000 STEPS WITH ADAMW ON ONE T4 GPU IN TWENTY-FIVE MINUTES · QUANTISED TO FLOAT16, 19.6 MB DOWN TO 9.8 MB · INFERENCE IN PLAIN JAVASCRIPT WITH A KV CACHE AT ROUGHLY 110 CHARACTERS PER SECOND · NO SERVER, NO API, NOTHING LEAVES THIS PAGE · THE MODEL LEARNED THE SHAPE OF A MATCH REPORT AND NOTHING ELSE: IT HAS NO CONCEPT OF A COUNTRY, A CALENDAR OR A PERSON · EXPECT DEFUNCT NATIONS, CITIES IN THE WRONG COUNTRY AND PLAYERS WHO NEVER EXISTED · EVERY RESULT IT PRODUCES IS INVENTED
MODEL AND PAGE BY ADEE1T · MIT LICENCE