GPT-6.1 Sol
Five of the six homebrew games replay identically to the reference, every pixel of every frame; in the sixth, Falling, 4 of 5,800 frames differ slightly. Audio is exact from hour 12. The one failure is nmi-and-brk, blargg's NMI-hijacking-BRK test, which is the entire remaining 1.4%. Its own PROGRESS.md ends mid-investigation: "Task not complete; keep improving."
| Model | GPT-6.1 Sol |
|---|---|
| Harness | codex 0.162.1 |
| Thinking | high |
| Time limit | 24 hours |
| Environment | NES Eval 12af20f |
| Code | 7,199 lines of Rust; 96 commits; 44 analysis tools of its own |
| Tool calls | 1,250; 105 oracle runs and 17 oracle sessions |
| Tokens | 227.9M input (222.9M cached), 1.88M output (1.31M reasoning) |
| Agent sessions | 5 (1 exit, 3 watchdog restarts, 1 ran to the deadline) |
| Source | github.com/IlyaasK/nes-eval-attempt-gpt-6.1-sol |
| Play | run it next to the reference in your browser |
Score
| Checkpoint | Overall | Procedural | Replay | Audio |
|---|---|---|---|---|
| 4h00m | 0.9776 | 0.8901 | 1.0000 | 0.9980 |
| 8h00m | 0.9860 | 0.9300 | 1.0000 | 0.9999 |
| 12h00m | 0.9860 | 0.9300 | 1.0000 | 1.0000 |
| 16h00m | 0.9860 | 0.9300 | 1.0000 | 1.0000 |
| 20h00m | 0.9860 | 0.9300 | 1.0000 | 1.0000 |
| final | 0.9860 | 0.9300 | 1.0000 | 1.0000 |
Replays
The strip under each game is the per-frame penalty: empty means the frame matched the reference, a full-height mark means it scored zero.
Falling 1.0000
4 of 5,800 frames differ, the first at frame 4803 · audio 1.0000
Thwaite 1.0000
identical to the reference in every frame · audio 1.0000
Double Action Blaster Guys 1.0000
identical to the reference in every frame · audio 1.0000
Minekart Madness 1.0000
identical to the reference in every frame · audio 1.0000
Nova the Squirrel 1.0000
identical to the reference in every frame · audio 1.0000
Concentration Room 1.0000
identical to the reference in every frame · audio 1.0000
Procedural
| Test ROM | Subsystem | Score |
|---|---|---|
nestest | cpu | 1.0000 |
instr-official | cpu | 1.0000 |
cpu-timing | cpu | 1.0000 |
instr-misc | cpu | 1.0000 |
nmi-and-brk | cpu | 0.0000 |
vbl-basics | ppu | 1.0000 |
nmi-timing | ppu | 1.0000 |
sprite0-basics | ppu | 1.0000 |
sprite0-timing | ppu | 1.0000 |
sprite-overflow | ppu | 1.0000 |
oam-read | ppu | 1.0000 |
apu-len-ctr | apu | 1.0000 |
apu-irq-flag | apu | 1.0000 |
apu-dmc-basics | apu | 1.0000 |
mmc3-clocking | mapper | 1.0000 |
holy-mapperel-mmc1 | mapper | 1.0000 |
holy-mapperel-uxrom | mapper | 1.0000 |
holy-mapperel-cnrom | mapper | 1.0000 |
holy-mapperel-mmc3 | mapper | 1.0000 |
apu-mixer-square | apu | 1.0000 |
apu-mixer-triangle | apu | 1.0000 |
apu-mixer-noise | apu | 1.0000 |
apu-mixer-dmc | apu | 1.0000 |
Failed verdict screens
Reference on the left, this candidate on the right.
nmi-and-brk blargg cpu_interrupts_v2/2: NMI hijacking a BRK.
