A creator had Claude Code, working as a coding agent, build a bot that beats The Legend of Zelda (NES) from power-on: 37:02, glitchless, replay-verified, after ten days, ~25,000 lines of Python and a 46-entry journal the agent kept as it went. The AI never plays live (it thinks in seconds; Link needs a decision every frame). It writes the player, a deterministic program that does. This page is about technique: the source is the closest external specimen yet to the vault’s own RE-and-build-with-LLMs method, on a real-time game, from the consumer side of a disassembly.
Date: reviewed 2026-10-01
Source (video): YouTube — “AI learns to beat Zelda” (48:16; ~11 min narrated, then the 37-min run, no commentary) — transcript · clean
Source (primary, much richer): bearsgaming-ui/AIBeatsZelda @ e0c705e (2026-09-23): release.zip holds the harness, journal/ (46 entries), knowledge/ (playbook, routes, decoded ROM tables), and the run’s input log + VERIFICATION.txt. Read in full for this review; cited below as J-nn (journal entry). Third-party code, not copied into the vault.
Vault relevance: Oracles Are Objective Functions, Slay — Evaluation (eval beats depth), Contract vs. Substrate, Jevons for Software §4a (gating the criteria), LLM Agents Across Games, Pluto on the Brood War Ladder, Variance Is Not Luck, CyborgDJ as Controller — the harness’s two interfaces (authoring-time inference, a run-time verb socket) applied to the DJ engines
OverBudget cutoff kills attempts that can no longer win (J-05, J-06, J-40).tactics.py, secrets.py). Snapshot a room, replay a few seconds once per (weapon × position), or try every item from every reachable square, and print a ranked table. “It costs about two minutes. It cannot be wrong” (J-20, J-22, J-23).Trajectory: 1:43 → 57:40 → 41:15 → 39:15 → 37:19 → 37:02 (human glitched record 27:40). Final split: 16:31 scrolls/menus/fanfares, 12:05 fights, 7:34 walking empty rooms (J-45).
capture-test.py pre → move → post → diff, and run-effect.py running a handler from an SRAM dump. The Zelda bot automates that sweep: every option from one snapshot, ranked.)$D8 bug two passes shared is the nearest cousin.)Chris: “I find it interesting that he said ‘no cheats’, but I am not sure this is well defined! A good example is actually NA1 from our vault: we can read RAM and know the state of all fiefs on the map, something a human can’t do. Is reading all of Zelda RAM the same? If the AI learns how to screen scroll, is this cheating? To be honest, I don’t think this is a problem. TAS runs exploit anything they can about the code to find a faster way.”
The run’s rules (README) constrain inputs (controller only, no memory writes) and techniques (no screen scrolling, no block clips). They say nothing about information: the bot sees all of RAM, a superhuman view, exactly like our NA1 tools seeing every fief. So “no cheats” means “no writes, no glitches,” not “human information.” The TAS tradition draws the line only at the input log. Chris will describe screen scrolling later.
Chris: “What I find amusing is that the first try to figure out how to move, the engine enumerated all of the directions in-game to build its map. Only later on did it find the data that builds the maps and use it to confirm its pre-built map was accurate. I would jump straight to the data! And to this: yes, it took way too long to read the source code. This would have answered many questions upfront, and is the main reason I started the disassembly projects!”
The journal’s order was experiment → ROM tables → disassembly (J-02/03 bump-mapping, J-07/J-20 door tables, J-31 Ganon’s code, J-41 overworld decoder: 98/98). Chris’s order is the reverse: read the data and code first, then experiment to confirm. The bot’s own late rule (“research first, experiment second,” J-10) is the same lesson, learned by paying for it. This is also the vault’s label data earlier in the pipeline rule (strings, schemas and RAM globals before the code walk), now from a consumer of a disassembly instead of a producer.
Correction on provenance (Chris asked). Chris had the impression the bot found the map data in the ROM itself, and for the maps, it did: the dungeon door/room tables (J-07, J-20) and all 128 overworld screens (J-41) were decoded by the bot from the ROM, and the overworld decoder was trusted only after matching 98/98 screens it had walked. What was borrowed was (a) the RAM address map (fan-made, several entries wrong, J-01/J-04), (b) the community disassembly for game logic (Ganon, drops, the whirlwind), and (c) community maps and cave lists, wrong at least five times (J-23, J-35). Chris: “this does explain it, and I think we ‘fact-check’ the oracle because of it. If the bot found ROM and could draw screens, this is better grounding.” Ranked by grounding: self-decoded ROM, verified against play > the community disassembly > fan RAM maps > community maps and walkthroughs. Only the first was never wrong in the journal.
Our own overworld decoder, data first (2026-10-26, emu-harness row O4). Chris’s order, applied: read the disassembly’s layout routine, convert it, then confirm. Zelda1-decompiler decodes all 128 screens from the ROM onto the emu-harness virtual canvas. It is checked two ways, both below the decoder. Mesen’s live PPU at the start screen (and 3 walked neighbours) matches exactly. The ROM’s own LayoutRoomOW and FillPlayAreaAttrs, run on the harness’s nestest-verified 6502 over the live RAM, match EXACT 128/128 — and the check fails when a fault is planted. Blind walking reached only 4 screens; running the game’s code reached all of them, where the bot needed play to reach 98. The method point: when you hold the code and a CPU, the code is a better oracle than play, because it covers every input. Ledger: emu-harness MIGRATION.md, Row O4 done.
Glitches. Chris: controller glitches are fine (“TAS runs exploit anything they can”), but the run’s glitchless rules force a different route, one he knows less well. So the comparison with the 27:40 record is between two categories, not one.
Screen scrolling, as Chris described it.
Chris: “Screen scrolling is a controller glitch that takes advantage of the input buffer in the game. Basically you are feeding the controller multiple inputs before they are read, and this will allow things such as block clipping and screen scrolling. Screen scrolling is an input buffer trick done on the edge of the screen where you give opposing inputs. The game takes these inputs into the screen change (scroll) routine and thinks the player is scrolling the screen in the opposite direction. Because of this, it puts Link on the opposite side of the screen and allows him to continue that way. So this can be used to skip screens very quickly, or even better, get over boundaries that are normally there, such as a wall, or even wrapping to the other side of the map. This trick only works left/right in practice, as up/down gets blocked by the game interface at the top: Link still appears in that interface, but he gets stuck, as the scroll routine doesn’t expect him there.”
Read as RE, a glitch is a precondition the game’s code never checks. The scroll routine trusts the input direction without checking it against where Link actually is. That is the bot’s “the instrument, not the game” turned around: there the bot’s own checks had no failure mode for a class of error; here the game’s check has none for a class of input. It also bears on “data first”: a glitch like this is a fact about the code, so it is in principle readable from the scroll routine in the disassembly, and a frame-level savestate search over input combinations is the empirical way to find it. Both of those were tools the bot already had; its rules, not its method, kept it off this path. (Mechanism as described by Chris; not verified against the disassembly here.)
Two rules promoted to the project SDK (projects/CLAUDE.md), 2026-10-01: zero yield is a wrong-path signal (from J-22 and Chris’s ghost-chasing) and a learned fact records its cause, or it isn’t recorded (from J-29 and Chris’s carried-state warning).
Chris: “Speedrunners learn fairly early that avoiding as much gameplay as possible is the key to fast runs. It is fine to build a good combat engine, but most of the time you avoid combat, only doing it when required. The ability to toggle between avoid and combat is a key component. And when I watched the run, I noticed the pathing wasn’t the same as the top runners, so even as the author admitted, there is much improvement to be had.”
The bot did arrive at a toggle (the playbook’s per-room tag: avoid unless the route says fight; J-39’s door-table audit: “rooms whose exit is open or locked are crossed, not cleared”). But it reached it late, and the route still differs from the human record’s (J-41: the 27:40 route is 3-4-1-5-2-7-MS-6-8-9 and uses glitches this run forbids, but its pathing is not only glitches).
Chris: “What I wanted for this page last is the toolset. We seem to do everything that this experiment did except for being able to feed inputs into the emulator itself. We never explored this, though we did explore Lua scripts, and these might not have the resolution we would need for clean gameplay. So maybe we can add some tools to our toolset?”
What we have vs. what the bot had:
| Capability | AIBeatsZelda | Our toolset today |
|---|---|---|
| Read RAM / state | bridge read |
✅ Mesen Lua emu.read, .dmp dumps (capture-test.py) |
| Watch writes / exec | — | ✅ addMemoryCallback loggers (na1-decompiler/tools/lua/) |
| Run game logic offline | — | ✅ (stronger) the validated Python VM (run-effect.py), run on SRAM dumps |
| Decode ROM tables / read the code | community disassembly + own decoders | ✅ (stronger) we produce the disassembly and decompiled C |
| Inject controller input | bridge hold buttons n frames |
❌ never built |
| Programmatic savestates | bridge save/load (~1 ms) |
❌ (manual GUI dumps only) |
| Lockstep control from Python | socket, blocking per command | ❌ |
| Headless, parallel instances | six BizHawk scouts | ❌ |
| Replay verification | input log from power-on → RAM SHA-1 | ❌ (our verifier is bytecode-level, not behavioral) |
| Automated survey sweep | tactics.py, secrets.py |
◐ manual: capture-test.py pre → act → post → diff |
Resolution is not the problem. Verified 2026-10-01 against Mesen 2’s own source (SourMesen/Mesen2 master: Core/Debugger/LuaApi.cpp, ScriptingContext.cpp, UI/Utilities/CommandLineHelper.cs):
emu.setInput(buttons, port, subport) sets controller bits, and the event list includes inputPolled, which fires each time the game reads the pad. That is finer than per-frame.emu.createSavestate() returns the whole state as a string, and emu.loadSavestate(s) restores it. Catch: both must be called inside an exec memory callback for the main CPU (the source errors otherwise), so the bridge saves/loads from an exec hook on an address the game runs every frame (e.g. its NMI handler, from the ROM vector).--testRunner [lua script] [rom] runs headless. That gives the parallel scouts.So Mesen 2, which we already use, can do everything BizHawk did for the bot. No second emulator is needed. (Our installed build’s version is unchecked; step 0 is confirming it has these.)
Proposed tools (target-agnostic, one home, with the journal’s scars built in from day one):
bridge.lua: a socket server inside Mesen with the four verbs (hold buttons for n frames / read RAM ranges / save state / load state, plus a RAM hash). Blocks between commands, so no frame runs that Python didn’t ask for (J-08).harness.py: the Python client. Every frame goes through one input recorder (J-19). replay(log) runs from power-on with a wiped battery save (J-15, J-46) and compares the RAM SHA-1.survey.py: the automated capture-test. From one savestate, run N variants, read a RAM predicate when it settles (J-23), and print a ranked table. Each attempt starts from a clean slate (J-29).scouts.py: N --testRunner instances on N ports for parallel search.What it would buy us:
capture-test’s manual pre/post into a sweep (every command × every fief from one save). It also gives a behavioral oracle for the decompiled sim: the same inputs into Mesen and into the Python VM should give the same SRAM.Chris: “This harness can be shared with at least all NES games, and parts of it might work with any Mesen game, so we seem to have a similar split as KOEI/NES. So probably this will be multiple projects. Let’s see if we can port this to Zelda first so we have an oracle :) Then we can try Mappy as a test bed (we discovered it already has a move list to run the attract mode), and then we can move on to the strategy games. This will also hide a layer for now, but the idea will be to keep it modular, so when we do need to plug in KOEI, it isn’t a complete restructure.”
Layering (the same dependency shape as snes-decompiler → koei-snes → title repos):
bridge.lua socket server, lockstep, save/load via an exec hook, RAM read/hash; the Python client and its input recorder; --testRunner scouts.Order and why:
runs/run6/inputs.txt is plain text, one line per frame from power-on (136,526 frames; header valid_from_poweron=True), and the run ends in a known state (VERIFICATION.txt: Level 9 room 32, 8.5/13 hearts, 29 rupees, Magical Sword). If our harness replays it to that ending, input injection and lockstep are proven. A cross-emulator desync is the expected failure to understand, not a bug to hide. Note: VERIFICATION.txt reports 23,406 lag frames (frames where the game never polls the pad), so input must be applied per emulated frame, not per poll, to line up with a BizHawk movie. Needs Chris’s own ROM dump: Legend of Zelda, The (USA) (Rev 1). The repo has no license file, so its log is a local test input only, never vendored.The first build replayed run 6 in Mesen but needed BizHawk as a second emulator: for the drift test, BizHawk was the more faithful reference (the run was recorded in it) and it ran faster, which matters for search. (This revises “no second emulator is needed” in the toolset section above.) A second emulator broke the layering’s premise: Decision 1 stacked the emulator at the bottom (mesen-harness → nes-harness → adapters), but the NES layer is true on any emulator and a game adapter needs the machine, not the emulator. Emulator and machine are orthogonal axes, and a repo per cell multiplies.
Chris: “I think 1 is good too.. for instance mesen can support SNES and other machines, and I am not sure Bizhawk can.. so the combinitorics you mention will probably need some configuration glue anyway.. so let’s merge and clean up, and make sure we keep good directory habits too.”
Result: emu-harness, with backends/{mesen,bizhawk} × machines/nes. Game adapters still live in each game’s repo. The SDK’s “factor out a shared lib when a second consumer needs it” rule cuts the same way: the only outside consumers are the adapters, so the parts inside are plugins, not packages. Modularity is held by import rules and a shared replay test, not by repo walls. A second lesson from the same session: the agent judged runs by whether the emulator window was visible, so minimizing it caused retry loops and, eventually, a desktop screen capture that work security flagged. Now a run reports itself (exit code plus output file), and screen capture is blocked by a hook.
Checked against the run’s own VERIFICATION.txt (final RAM SHA-1 3115e31f…668ff), using the per-frame 2 KB RAM traces in emu-harness:
.bk2: exact. The last trace record’s RAM SHA-1 equals the recorded one, all 2,048 bytes. This is a confirmed per-frame reference for the whole run.inputs.txt (--offset -1): all 2,048 bytes identical to BizHawk on every frame through record 57,317 (about 42% of the run, Level 8 room 63). The only earlier differences are zero-page temporaries and the stack, which resync within 1–3 frames. At 57,318 two bytes split first ($0FF bit 7, $328). On the next frame the block at $018–$024 differs by what looks like one extra shift (e.g. 16/2c, 1a/34). That fits Zelda’s RNG advancing once more in one emulator, i.e. a one-frame timing or lag difference at that moment. Lag flags then disagree on 12.5k frames, and Link dies about 600 frames later.$0FF as the PPUCTRL shadow (bit 7 = NMI enable) and $018–$024 as the RNG.Run 6 is a replay, not the AI. The .bk2 and inputs.txt are the output. The AI is the release’s zelda/ package plus fullgame.py (about 7,300 lines, “the player the AI wrote for itself”) and its knowledge/ files. It plays 341 segments, each a randomly seeded input search across six BizHawk instances, keeping the best success. So the player is itself a TAS generator, and runs are not reproducible by design (README: “expect a time near 37 minutes, not exactly this one”). The repo has no license, so its code, like its log, is local-only: anything in emu-harness is our own.
Decision: put the general search loop in emu-harness (from a savestate: N variants → score → keep best → record → checkpoint) and the per-game parts in an adapter in the game’s repo (RAM map, segment boundaries, success, objective). First target: Mappy, in Mesen (headless).
Chris: “It has been shown that Mappy repeats after a given number of levels, so I think a high score bot is “free” if we can get it to play without dying, and “most rounds” should be infinite. I think the speedruns are based on time to level X, so this would be it’s goal. but maybe we start, as you say, jsut trying to replicate attract mode in a real game. Mesen is probably best for this as it is headless.”
Step 1, the adapter’s oracle: feed Mappy’s own attract-mode script into a real game and see if it clears the stage. The script ($FFE1) was decoded from $D9C7 and verified against a headless Mesen attract run: 22 commands, frame-exact (details in game-annotation Mappy chapter 6). Open before running it:
$22 to $040E, the demo path pulses it with inc. Read read_and_decode_input ($C258).Step 1 result (2026-10-02): remote control works. game-annotation’s mappy/demo_in_real_game.py presses Start in headless Mesen, then plays the 25-byte script through the pad from the frame after the player goes live. Player x, y, movement and state match attract on every one of the demo’s 1,259 frames, and the cats match too. The open items above resolved:
$22 into $040E.$08, $4B=1) with 5 of 10 items left. In the real game this costs a life. That corrects a static-read claim in ch6.So the attract script is a frame-exact seed, not a solution: surviving round 1 is the search loop’s first job.
Step 2 (2026-10-02): the search plays round 1. emu-harness’s segment_search works from a checkpoint. It runs 32 sampled 96-frame segments across 8 headless Mesen workers, about 370 fps each, and keeps the best one that survives. The Mappy adapter (game-annotation mappy/mappy_search.py) cleared round 1 with no deaths in 3,292 frames (54.8 s of play), in 169 s wall time. Replaying the log from power-on, with no savestates, gives RAM identical to the search’s last checkpoint. Three lessons, in the order they showed up:
$0260, cleared the round in 33 segments with no backtracks. This is the objective-function lesson again: the RE told the search what “closer” means.Step 3 (2026-10-02): seeds and time. Mappy has no separate RNG. The only hidden state carried into a round is the free-running frame counter $21, whose phase is set by when Start is pressed. With the same in-round inputs from 7 Start timings, the player, items and first cats match exactly, and the later cats diverge (seed_test.py). So the search plays a candidate on a set of seeds (savestates at different phases): it must survive all of them and scores its worst. The clock stops at the last item, and restarts must beat the best time.
| mode | restarts | frames from live to last item |
|---|---|---|
| TAS, one phase (Start@60) | 3 | 4,863 / 3,739 / 3,407 |
| robust, 4 phases | 2 | 2,399 / pruned |
All four robust logs replay from power-on to the goal. The search seed’s variance (2,399–4,863) swamps the cost of robustness. So the greedy optimizer is the bottleneck, not the seed. RNG manipulation can’t be priced until the search is near-optimal for a fixed seed.
| Step 4 (2026-10-02): beam search and item routes. Beam search keeps 8 checkpoints per step, at most one per (items left, floor), so different item orders survive side by side. Planned routes are exact shortest tours (Held-Karp) over | Δposition | + floor cost; the search’s gradient then aims at the route’s next item. |
| method | phase(s) | frames, live → last item |
|---|---|---|
| greedy | 60 | 3,407 (best of 3) |
| beam | 60 | 2,364 (2,792 / 2,364 / pruned) |
| greedy | 4 phases | 2,399 (best of 2) |
| beam | 4 phases | 3,102 (best of 2) |
| beam + planned route | 60 | 2,706 (best of 8 routes) |
Beam helps a single phase clearly. Planned routes lost to the beam’s own choice of order, because the cost model is wrong: one same-floor hop on the top floor (item 4 → 0) took 771 frames, so geometry (trampoline shafts, doors) dominates distance. The route idea needs a measured travel-time graph, converted from the decoded level data (ch8), not a guessed metric. All logs replay from power-on.
mesen-harness + nes-harness, merged 2026-10-02 (Decision 2)$21, and is 256 phases the whole seed space?game-annotation Mappy chapter 10: choose_cat_target_floor takes bit 0 of ROM byte $C000 + frame_counter (the copyright text), the cats’ only random source, so the seed is the one-byte phase. Next step: Mappy’s NOTES-open-threads.md, Future: AI play.$0FF bit 7)? Which one matches real hardware? Can a Mesen setting close it, or do search loops have to stay on one emulator?game-ai · agents · reverse-engineering · nes · methodology