Brotato RL
A reinforcement-learning environment for Brotato (Demon, Danger 5, endless mode), built in a 12-hour hackathon with PufferLib 5. Code: github.com/IlyaasK/brotato-rl.
Play it
The same C simulator, compiled to WebAssembly, playing a 10B-step Demon endless Danger-5 policy at 60 Hz. Click the canvas first so it takes the keyboard: WASD / arrow keys drive the same 8-way action space, so you can take over from the policy at any moment. Open full window.
What it is
- Simulator: a C reimplementation of the game that runs on CPU and CUDA. GPU and CPU results are bit-identical on every state field (20,000 steps × 32 envs, every field compared, padding included).
- Checked against the real game: a Godot mod drives the real game headless and records every frame. The simulator replays those recordings and matches value for value: the discrete half of the 203-value observation (masks, counts, ids, levels, wave flags) is exact over 827 ticks — 0 of 130,666 values differ — and the continuous half is within 1 float32 ULP (max 9.15e-07, none above 1e-06).
- Honest bound: the game does not reproduce itself. Two runs with identical seed, pack and actions diverge from each other by 0.0656 in observations and 3.29 px in enemy positions (and by up to two waves in outcome), so our agreement with any one run is far tighter than the game's agreement with itself.
- Full action space: 31 actions covering movement, buying weapons, items, pets and structures, reroll, lock, ban, recycle, combine, level-ups and crates, with a per-phase action mask. The observation has 203 values.
- Training speed: 2.2M env steps/s at 4,096 environments, rising to 3.9M at 6,144–7,168 on one RTX 5070; a 1B-step run takes 16 minutes, the 10B-step policy below took 3h20m.
Results so far
| Result |
| Real-game recordings, matching frame for frame | up to ticks 1977 / 1551 / 1854 (of 3,600 / 3,600 / 2,799), every recorded action matching |
| Policy in the simulator | reaches wave ≈3 in the training sweep after 1B steps; 6 weapon slots filled, buys weapons, items and pets |
| Same policy in the real game (10B steps) | 4.33 waves on average over seeds 1–3 on both the mid and final checkpoints (best 6), with the simulator matching the game's wave count exactly on 8 of 11 seed pairs |
| Reward | +1 per wave completed, plus potential-based shaping from the endless-mode guide: harvesting and luck first, the pruner → potato thrower → excalibur weapon ladder, Max-HP scaling (materials deliberately unshaped, since paying for holding gold fights the snowball) |
Earlier clip
A 1B-step policy playing the desktop simulator, before the browser build.
Notes
- The sprites in the demo above were extracted from the author's own copy of the game; the simulator and its repository ship no game art, and they build and run with primitive-shape fallbacks when it is absent.
- No animation data is in the numeric core: each enemy and weapon draws a single frame, so walk/idle/death clips are not played, and per-kind sprite offsets are not applied (tall bosses sit slightly lower than in the game).
- The browser build is single-threaded, which is irrelevant for play.