How a game-playing robot coded “Super Mario Maker” onto an SNES—live on stage

DULLES, Va.—Regular watchers of the annual Awesome Games Done Quick (AGDQ) video game speedrun marathon are probably intimately familiar with the power of TASBot (short for tool-assisted speedrun robot). Two years ago, the emulator-fueled bot used its controller-port interface to write a simple version of Pong and Snake on top of a running Super Mario World cartridge. Last year, TASBot outdid itself by using a copy of Pokemon Red and a Super Game Boy to force a live, IRC-based Twitch chat through an unmodified Super Game Boy.

By now, simply taking over a game and replacing it with a brand new app was beginning to feel a little predictable. So this year, TASBot decided to show off a new skill. At the AGDQ marathon, the bot set out to edit new features onto a game that’s still running in active memory. TASBot wanted to be magnanimous with its new capabilities, too, allowing human players (and livestream viewers) the opportunity to edit the game on the fly.

But just how did TASBot—and the team of coders behind it—intend to turn an old game of Super Mario World, running on a standard SNES, into a heavily editable game of Super Mario Maker? Luckily, we had a behind-the-scenes invite to the event and the opportunity to find out.

The setup

The archived Twitch video of TASBot’s SNES “Super Mario Maker” exploit looks like magic; or at the very least the kind of thing that requires a hacked ROM or memory manipulation through a Game Genie-style device. But it’s important to remember that like in the past, everything TASBot does is technically possible on standard classic game hardware and software. That is, it’s possible provided you have the superhuman ability to enter precisely timed controller inputs 60 times a second.

The first step to hacking up a level editor on top of Super Mario World is old hat for TASBot by now. The robot takes “total control” of the system through arcane in-game glitches. Next, it juggles items with pixel-perfect positioning and split-second timing to effectively write and execute a bit of assembly code through the system’s active sprite management memory.

This year’s takeover uses a slightly different total control method than those in years past. The new route uses precise pausing and unpausing after getting hit by a mole enemy to get some in-game timers lined up just right to represent that crucial memory code. The end result is the same: TASBot tells the game to start reading controller inputs as raw, binary programming code at a rate of about 3.8KB/sec.

From there, it’s relatively easy for TASBot to use precisely timed button presses to essentially write a new program that lets it take “total control” of the SNES. The details involve first writing a block loader to a small, “safe” area of memory, then running that block loader to continually sample the controller inputs for new data (the process is described in much more detail in this write-up of the 2015 Pokemon Red/Super Game Boy exploit).

The total control process was made simpler for the TASBot team this time around by a handy Lua script that can recode arbitrary PC files into the appropriate controller inputs. “For ‘Pokemon Plays Twitch,’ when we wanted to test a new version of the payload we were reliant on Ilari to create a new movie file,” TASBot organizer Allan Cecil (who goes by DwangoAC online) told Ars. “For this version, Ilari created a handy ‘script kiddie’ Lua script that allows us to specify a binary file to write to a particular location in memory based on its filename.”

Hiding in the SRAM

In previous demonstrations after TASBot gained total control of a system, it threw out the existing game and coded whatever software it wanted in its place—Pong, Super Mario Bros., Twitch chat, etc. This year, the level-editing script the team was building would be inserted on top of the still-active Super Mario World code already in memory. That way, instead of having to recode an entire Mario engine, the code could simply call existing functions for everything from block placement to screen scrolling to Mario physics. As Cecil put it, “Hey, why write something new when you don’t have to?”

As you can imagine, keeping the real Mario code in active memory while writing a level editor on top requires a much lighter touch, and much more delicate memory management, than in the past. “There’s not much main [SNES] memory that’s left over once Super Mario World gets done hogging it,” Cecil said.

As it stands, TASBot can only write over a few small, “safe” portions of the system’s 128KB of main RAM without breaking the copy of Super Mario World already loaded there. What’s more, that safe RAM is laid out in small, scattered pockets that are hard to access contiguously (and the much larger ROM chips on the cartridge itself can’t be overwritten at all).

The TASBot team briefly considered trying to write and execute the level editor on the RAM dedicated to the SNES sound chip, but Cecil said that idea was thrown out pretty quickly. The sound system was designed to run totally independently from the graphics and logic systems powering the rest of the SNES, so trying to hardcode and run executable logic on there “goes against everything the SNES was supposed to do” (even more so than everything else TASBot does, apparently). “Accessing [the audio RAM] is just so actively risky and dangerous. You’re just asking for trouble,” he said.

The safest place to store TASBot’s inserted code, it turns out, was on the game cartridge itself. Specifically, the small bit of rewritable SRAM that many SNES games include for storing battery-backed save games. Overwriting that would ruin any saved progress on the cart itself, but it wouldn’t alter the running Super Mario World code at all.

While the original Super Mario World cart has a mere 2KB of SRAM to work with, Nintendo later released an expanded cartridge that put the Super Mario All-Stars compilation and Super Mario World in a single package. The expanded Super Mario All-Stars + Super Mario World (SMAS+W) cartridge has a whopping 8KB of SRAM to accommodate save files for all of those games. TASBot would end up needing almost all of that expanded space to write its level editor.

Fooling the game

Even with a convenient place for TASBot to write new code, it was still difficult getting a level editor running on top of a game that was decidedly not designed to have a level editor running on top of it. “Painful” is the one word description of the takeover process used by coder p4plus2, a senior at Cal State East Bay who did most of the work on the actual level editor payload.

“It’s been one of those interesting experiences,” he told Ars. “You never know what you’re going to be able to get into the final product… It was definitely a lot of last-minute panic, a lot of sleepless nights…”

The first step for inserting a level editor into Super Mario World is convincing the game that the new code belongs there in the first place. “The thing that has to happen is you’re kind of running a game loop inside of a game loop,” p4plus2 said. “You’re running your editor as an outer shell, and you take over the internal game and sort of restart the game inside of itself to jump into a known state—you have to reinitialize things to something you know you can control.”

A lot of p4plus2’s further coding efforts focused on “stopping the game from acting like it wants to act” as the player is editing a level, he said. For instance, Mario touching an enemy while the level was being edited would lead to instant death and the end of the level. “We needed to come up with loopholes like hiding him off screen, so he doesn’t get hurt,” p4plus2 said.

The level editor code is full of similar routines that make sure the game doesn’t break itself while being edited. The game’s entire sprite-handling system had to be reverse engineered to allow the editor to let the cursor move in-game sprites without them being actively processed by the game engine. The usual pause function also had to be overridden so the start button could restart the level, instead. Mario’s standard return to the world map after death had to be completely circumvented as well.

There were also ever-present worries about introducing too much lag into the game. If the level editor code takes too many processor cycles between display frames, the carefully timed inputs from TASBot could end up becoming desynchronized from what was planned, breaking the entire process. P4plus2 estimates his code leaves only a few thousands extra CPU cycles of overhead to work with each frame, under normal conditions.

If a human level maker places too many enemy sprites on screen at once, those extra cycles can easily spill over into game-breaking lag. As it is, the team spent many of the final hours before the live demo scrambling to reposition particularly processor-intensive enemies, and recoding inputs so blocks didn’t end up desynchronized out of place. The final version of the level was sent to TASBot literally minutes before the team had to walk down to the stage.

The best laid plans…

Even with SMAS+W‘s “expanded” 8KB of SRAM, there wasn’t much space to code a truly complete level editor. Some desired features for the editor had to be thrown out so the core of the editor could fit, the team said. Simply displaying the names of each sprite in the editor interface, for instance, would have required a text table about 1KB in size, p4plus2 says, eating up 1/8th of the usable coding space in one fell swoop. Features allowing for players to edit their own sprite images or change the colors of existing items were also thrown out due to space concerns.

Other features were shoehorned in only by throwing out unused game features sitting in the system memory. To allow the editor to restore a level to its original state after Mario dies, for instance, p4plus2 ended up overwriting memory that previously held the game’s map screen. That’s a perfectly fine hack… as long as you never want to use the map screen again. “If you want to restore to the original game, you can’t do so,” he said.

Then there are the desired features that were obstructed by sloppy coding on Nintendo’s part more than two decades ago. The TASBot team originally planned to let the editor use graphics and items from all five of the Mario games on the SMAS+W cartridge. But “it turns out [all the Super Mario All-Stars games], when they ported them over, they did a pretty haphazard job and barely got them working,” Cecil told Ars.

This haphazard coding means the SMAS+W cart uses some inconsistent memory management for the in-game sprite tiles, putting enemies and objects in highly unexpected places in the code. “They’re unstable… there were a lot of hacky nuances where they had really unusual spaghetti code,” Cecil said. “They just threw things at the wall and hoped it would stick.”

With enough coding time, p4plus2 said he could work around that odd memory usage to get at the old object tiles. For now, though, he’s hoping the community will be able to shoehorn these other editor functions once he makes the code behind the hack public. In fact, he says he’s left 1 KB or so of overhead in the SRAM package just for this kind of future coding work.

The live demo

The live SNES “Super Mario Maker” stage demo, as it appeared to Twitch viewers

The honor of being the first people to edit a new level onto a stock Super Mario World cartridge went to a couple of popular Mario streamers. SethBling and PangeaPanga worked together to hardcode a tool-assisted movie file to automatically build a level as soon as TASBot took control of the game. The pair decided to recreate a castle level that was part of an exciting blind Super Mario Maker race earlier in the marathon.

After a few robot synchronization problems on stage, that level was rebuilt live in Super Mario World to the delight of 1,400 AGDQ attendees in a Dulles, Virginia hotel ballroom (not to mention many more Internet onlookers from the Twitch steam). It was soon up to another popular Mario streamer, dram55, to prove that the level was actually playable (he beat it without much trouble at all).

As a closing act, TASBot then gave its editor to the world… or at least the part of the world watching Twitch. Echoing experiments like Twitch Plays Pokemon, TASBot hooked its input script through an IRC interpreter, letting Twitch chatters make their own level. Button commands typed into Twitch chat were translated into controller code that would move the cursor and place a pre-selected set of four blocks.

P4plus2 told Ars before the live demo that he thought there was a good chance Twitch’s torrent of utterly random inputs would simply crash the game. However, what actually happened was much more entertaining. After a few false starts, the Twitch chat actually managed to corrupt the game to the extent that the entire level became a glitchy, cross-hatched mess of unrecognizable, almost unplayable blocks.