Look ma, no humans —

Meet MASHBot, the touchscreen-tapping, Nintendo DS-playing robot

New bot shows automated play doesn’t need emulators or controller-port “cheats.”

Meet MASHBot, the touchscreen-tapping, Nintendo DS-playing robot

Since making its public premiere at 2014’s Awesome Games Done Quick marathon, TASBot (the tool-assisted speedrun robot) has repeatedly amazed audiences by performing seemingly impossible in-game feats. Using nothing but pre-recorded electrical signals sent through a game console’s standard controller ports, TASBot has done everything from beating Super Mario Bros. 3 in under a second, to running Twitch chat through a standard SNES, to coding an SNES version of Super Mario Maker on the fly.

But no matter how amazing TASBot’s performances are, there’s still a group of naysayers out there that argues that the robot’s direct connection to the controller port makes the whole thing inauthentic, somehow. "Every single YouTube video I post, there's at least one guy calling us haxxors and saying we are filthy cheaters,” TASBot team manager Allan “DwangoAC” Cecil told Ars at the recent Awesome Games Done Quick charity marathon (AGDQ). “No matter how many times we explain that it's not a ROM hack, people assume that we've hacked the game, when we haven't, in the sense of changing its ROM."

So this year, in an effort to prove the doubters wrong, Cecil and the TASBot team set out to create “a replay device that’s the most insane thing we’ve ever done,” as Cecil put it on stage. Rather than just sending signals through the controller port, MASHBot (the Machine-Assisted Speedrun Hardware robot) can actually manipulate the controller itself (in this case, a Nintendo DS touchscreen), without any human intervention.

“What would happen if we could actually press actual buttons so we could take out every possible excuse about us not doing things legitimately?” Cecil said of the thought process behind the move. "If you can actually build a device that is physically pressing the buttons, there's not really any place you can go with that other than, 'We are just playing a game here.'"

“...we don’t want to punch a hole through the controller”

Well before MASHBot made its successful debut tapping on a Nintendo DS at AGDQ last Friday, the device started as an effort to play games on standard Game Boy Advance SP hardware. To enter inputs without the use of a controller port (and without emulators or console add-ons like the Gamecube’s Game Boy Player), MASHBot creator FunkmasterMP set up a system of servos for each button and directional input on the Game Boy itself.

But creating a robot that can reliably push buttons with frame-perfect accuracy (i.e., inside 1/30th of a second for a full press-and-release cycle) was harder than expected. “Basically [the actuators] weren't fast enough to do it,” Funkmaster told Ars. “When you try to get them to speeds where they could do it, they didn't push hard enough on the button.”

Cecil expanded on the hardware design problem. “As you increase the amount of torque, the amount of pressure you can supply increases, but we don't want to punch a hole through the controller,” he said. “At a certain point your actuator is not able to press and release and return to a place where it can press again fast enough."

Mashing the actuators at TAS-level speeds “quickly becomes very unreliable,” Cecil said. “It's very hard on the actuator [which] starts to heat up and then its characteristics change. Boy, I can't imagine why!"

The TASBot team was eventually able to get a prototype that could guide Link to the sword in The Legend of Zelda: Link’s Awakening. Shortly after that point, though, the actuators would desynchronize from the carefully timed emulator recordings, making a complete game run on real GBA hardware unworkable. So, with three months to go before AGDQ, Funkmaster decided to make a last-ditch effort at a physical speedrun robot, transitioning from GBA button presses to DS touchscreen taps.

Everything that can go wrong...

A hobbyist creator with a degree from the University of Ontario Institute of Technology, Funkmaster says he wasn’t prepared for all the issues he’d run into when he started on the tight-deadline MASHBot construction. “I’ve learned so much during this… I didn't know what I was getting into. Allan tried to warn me so many times. He was like 'hardware is a bad idea!'”

The first model of MASHBot used polystyrene foam core to support the system. “It was pretty decent, but one problem we ran into was sagging,” Funkmaster said. “Foam's not exactly the most sturdy thing.”

As if that wasn’t bad enough, Funkmaster says the motor lubricant he was using was eating through the foam supports. “That fell through in the most literal sense."

After converting to a more resilient wood frame, Funkmaster also replaced the assembly’s original CD drive motors with more powerful and accurate NEMA17 stepper motors, as you might see in a 3D printer. Funkmaster also didn’t have a DS stylus handy when he started, so a filed-down LED had to fill in for a time.

Though Funkmaster says MASHBot worked perfectly during home testing in Ontario, when he got it down to Rockville, Maryland, for the AGDQ event, he quickly found that “I plugged it into the wall and it wouldn't even boot. I had to do some very makeshift solutions, some I feel very awkward and almost regrettable that I've done, but it worked.”

Through additional testing over the marathon week, Funkmaster found himself in a bit of a Catch-22 situation regarding the sensitive cork, spring, and metal assembly used to actually push the stylus onto the screen. With each test, the actuator would wear out a bit more, desynchronizing tests that had worked in the past. That necessitated more testing, which led to more wear and tear, which then required more testing, and so on.

By the morning of MASHBot’s premiere, Cecil admitted that “the reliability of this part is concerning us for tonight. We don't have a lot of time to improve it." But the worrying wasn’t necessary, as MASHBot ran through a run of Super Scribblenauts without serious issue on Friday night. Twitch viewers even got to vote on which set of preselected wacky solutions the robot should perform in-game, as additional proof that this was all being run live from the streaming stage (and leading to this incredible spike in Twitch chat use, which seems to have given Amazon' servers some trouble).

MASHBot's AGDQ debut from a Rockville, Maryland, ballroom Friday night.

The MASHBot of the future

MASHBot can move from one corner of the DS touchscreen to another in roughly one-third of a second. That’s pretty fast, but Cecil admits it’s not quick or precise enough for a reflex-heavy game like Kirby’s Canvas Curse (Super Scribblenauts has a much more forgiving timing window for its inputs). "A human can go much faster and with much more precision,” he said. “A human has so much control. We are so lucky to have hands. Functional joints and opposable fingers and thumbs are such a great invention."

To that end, Funkmaster says he ultimately “wants to see this thing have a hand… just have a controller and two hands and see if it can play a game. Ultimately I want to have it [generalized], so you could have a SNES controller, you could put a Gamecube controller... that's why I want to go eventually to the hand, because you have a lot more dexterity.

“I want to show off that it's not just a tech demo anymore. I want to show that we're actually able to play games and just to show the power of technology.”

Cecil has even broader goals for MASHBot’s physical future. He says he’s been a bit inspired by Bally/Midway’s Addams Family pinball table, which features an automatic flipper feature that can hits the ball with perfect timing every time. With the aid of the ball-tracking technology on Multimorphic’s P3 pinball tables, Cecil says a future version of MASHBot would sure play a mean pinball.

We're looking forward to the day when a MASHBot successor can actually watch a gaming screen and react with faster-than-human button presses that make pro-level speedrunners look like amateurs. When gaming's Deep Blue vs. Kasparov moment comes, we may look back on TASBot as the first step toward humanity's obsolescence in gaming.

Channel Ars Technica