Nobody trained these weights — and they run a Nintendo

In the early hours of October 31 (Beijing time), “Show HN: A working Nintendo in ~1GB of Qwen weights” reached sixth place on the Hacker News front page: developer Drew Breunig's nesformer repo compiled the NES's 6502 CPU (the core of its 2A03), 2 KB of work RAM and the whole cartridge program ROM into a 466.6M-parameter checkpoint. The repo's one-line statement names both the method and the nature: “Nobody trained these weights — a compiler computed every one.” The model works by “predicting” the CPU's execution log one token at a time, while a small host program completes the console around it: drawing the picture (the PPU renders scanline by scanline), reading the controller, injecting the NMI interrupt at vblank. In the repo's demo animation, Super Mario Bros. World 1-1 walks, jumps and clears pipes — and per the repo's records, every frame matches a real emulator exactly.

The checkpoint is a borrowed shell: Qwen3_5ForCausalLM format that stock transformers loads with no custom code; 7 Gated DeltaNet circuit layers plus 1 all-zero attention layer, hidden 768, vocab 263,808, head shapes carried over from Qwen3.5-4B (16 key heads, 32 value heads, dim 128). It holds 466.6M parameters (933 MB in bf16), of which only 9.3M are nonzero — 3.3 MB zipped: most weights are zero because this is a machine drawn into the weights, not learned. One clarification: nesformer does not mean Alibaba's Qwen3.5-4B learned to play games; the official Qwen3.5-4B release page was not obtained this round, so the site's model catalog adds no entry yet — the gap is stated as-is.

How did this come about? The author is frank in the HN top comment: Opus 5.5 built the working weights and code with two prompts. The first asked it to read the Qwenputer article and source and assess whether the technique could emulate the NES chip and RAM, at what model size, and whether it could approach the standard clock — the charts it returned said any model would be slow; the second was “let's build it — slow is fine”. The author also names the decisive ingredient: the emulation community's long-accumulated conformance tests (nestest's golden log, 3,000-plus instructions) provided a clear acceptance standard; together with Qwenputer's ready compiler, this was a task with complete reference materials and crisp acceptance — exactly the shape where AI-assisted development shines.

The foundation of the whole method is a “state-as-memory” trick. In hybrid architectures like Qwen3.5, 3 of every 4 attention layers use Gated DeltaNet — a linear RNN — as the attention mechanism; each head keeps a 128×128 state matrix updated by the delta rule at every token. The Qwenputer author found that setting the gate to 1 (nothing fades), write strength β to 1 (full replacement instead of blending), and using one-hot keys (pointing at exactly one row) makes every write overwrite exactly one row and every one-hot query read one row back — “every row is a memory cell.” The cartridge, registers and RAM all live in those 128 rows. That linear attention with the delta rule works as associative memory (“fast weights”) was shown by Schlag et al. back in 2021; what's new is making it exactly addressable inside a production architecture, reliably enough to run a whole CPU.

Qwenputer came first (its blog is dated October 22): the author compiled first CHIP-8, then an RV32IM RISC-V CPU (plus the Zba extension and a bignum unit) into a real Qwen3.5-4B — 11 compiled layers and wider dims added around untouched original weights; the model routes arithmetic to the CPU internally via the tool-call format, with no external tool calls. The numbers are persuasive: NUPA long-integer arithmetic went from 48.6% to 100%, matching a Python tool; DeepMind sets div_big 97.5% (base 72.5%) and mul_big 97.5% (base 55%); 18M instructions free-running without error. There was even an experimental filesystem: 512 files, about 53,000 tokens, stored in DeltaNet state, with cross-chat facts at 90% (30% when pasting old chats back). The blog also previews two safety directions: models hiding reasoning in RNN state to evade CoT monitoring, and compiled weights as monitoring tools the model cannot switch off.

nesformer's division of labor: inside the model live the 6502 core (all 151 official opcodes, every addressing mode, flags, the stack, BRK and NMI entry), 2 KB of work RAM and the 32 KB PRG-ROM (stored twice, for instruction fetch and data reads); the host program handles the PPU (scanline rendering and palette), controllers, vblank NMI injection, and merely records APU register writes without sound. The host never looks at registers or RAM directly — it only eavesdrops through the execution log. The 7 circuit layers divide the work: layer 1 anchors the PC and registers/flags, layer 2 advances the PC, layer 3 fetches and decodes opcodes, layer 4 computes effective addresses, layer 5 does RAM/ROM reads, layer 6 writes results and flags, layer 7 emits the next token. Most instructions take a single token; real game code averages 1.1–1.3 tokens per instruction; the PC is never written directly, being implied by three counters. nesformer reuses Qwenputer's compiler — only the “circuit” changed to the 6502.

“Runs” and “runs correctly” are two different claims, and the repo answers the latter with a four-layer verification chain. Layer one: the reference CPU (nesformer/cpu.py) matches nestest's golden log for 3,348 instructions and py65 across 5.7M instructions of game code. Layer two: the compiled circuit (evaluated in Qwenputer's evaluator with its bf16 exactness checks) predicts every reference token — all of nestest plus 283,610 positions of two games' boot sequences. Layer three: the checkpoint in stock transformers (bf16, CPU) shows no mismatches across 2,651 positions. Layer four: free-running — the model consuming only its own outputs, with no reference machine to correct it — nestest's 4,586 tokens match and end in the expected HALT; Thwaite check 700,000 tokens/271 frames identical; Thwaite run 1,916,527 tokens across 600 frames with all 300 saved frames matching; Full Game run 2,645,885 tokens across 420 frames with all 210 saved frames matching.

The price is speed. The default engine (numba-compiled, one CPU core) runs about 1,650 tokens/s, the numpy engine about 210, stock transformers about 10 — while a real NES needs roughly 700,000 tokens/s, making the whole thing 150–290× slower. The repo's idle-loop skip (advancing the clock when a wait loop repeats with no writes) saves 2.6–4.2× tokens with frames unchanged; even so, a frame takes 1–6 seconds, and 600 frames (10 seconds of NES time) take about 25 minutes. The limits are listed plainly in the README: NROM (mapper 0) carts only (the class of Super Mario Bros., Donkey Kong, Excitebike) — bigger games use bank switching, not yet supported; official opcodes only; code must run from the cartridge ROM; no IRQs; PPU/APU not in the weights — no sound, mid-scanline effects not reproduced, sprite-0 hit and overflow approximate.

One point is easy to misread: in the demo, Mario “plays” because a controller script tells him to. The repo's run command spells out `--buttons "start@60-65,right@200-600,a@250-265+…"`, hard-coding Start, hold-right and jump timing; the model's job is executing “the machine itself” (CPU/RAM/cartridge) exactly — not player intelligence. The “~1GB” in the HN title refers to the 933 MB bf16 file — with only 9.3M nonzero of 466.6M parameters, it zips to 3.3 MB: this “console” is small enough to ship inside the repo. Another technical nuance: nesformer supports BRK and NMI inside the CPU but has no IRQs (the APU frame interrupt and DMC don't work).

Separating “trained” from “compiled” is the first reading. We're used to weights coming from training — parameters learned from data. nesformer offers another answer: weights can also be computed. In the HN thread, some re-raised the 2014 Neural Turing Machines paper, calling this an old story of “learning to use memory”; others countered that NTM trains a network to learn how to use memory, whereas Qwenputer/nesformer doesn't train “how to use it” — it compiles it. Exact compilation versus learned approximation is precisely the difference. Some also reach for the “Tetris in CSS” analogy, calling it a stunt. These are community views for contrast; we don't referee them. What's certain: of 466.6M parameters only 9.3M are nonzero (3.3 MB zipped) — the machine is small enough to ship in full, and it reopens the question of what weights can be.

The second reading concerns AI-assisted creation. The author's two prompts worked because three things stood around the task: a ready compiler (Qwenputer), the emulation community's conformance tests (nestest's golden log), and a crisp acceptance standard (frame-level parity). The author calls this run a model case of “complete reference materials and clear acceptance.” The HN thread reads it as a lesson: model performance rests on “high-quality reference materials and ready acceptance standards” — good task design is not just a prompt; it includes building acceptance into the task. This is not an isolated case: yesterday's Model Spec update and last week's Retell and Anima pieces showed the same rule — deterministic flows and clear acceptance are what make model outputs stable. Here it holds at the level of weights.

The third reading makes the efficiency gap concrete. Being 150–290× slower is not an engineering flaw but the method's own price: generating a CPU's execution log token by token — 1.1–1.3 tokens per instruction, a 700,000-token/s demand against a 1,650-token/s supply. Someone in the HN thread computes that running an OS this way would cost millions of times more than ordinary programs (a commenter's estimate, not a paper figure). The point is not to replace emulators — the work demonstrates two directions at once: for the real Qwen3.5-4B, the Qwenputer route has the model hand arithmetic to its internal CPU, with NUPA 100% and 18M error-free instructions showing the road works; for safety research, compiled weights can become monitoring tools the model cannot switch off. Perhaps the most practical reading: weights can be written and also computed — and to know what a model can do, look at how it is constructed.