NESBot: Tool Assisted Speedrun On Real Hardware

Xistic writes "For many years tool assisted speedruns were purely theoretical and the domain of emulators. No longer! Using an Arduino Duemilanove microcontroller to drive an actual Nintendo console, pjgat09 plays back prerecorded input to beat Super Mario Bros. in record time. The selection of possible games is limited: 'If the game relies on any uninitialized memory for randomness, or if it is heavily based on console timing, it may not work. In the case of Super Mario Bros however, as long as the button presses start play back at the right time, the movie will play back correctly.' The author includes complete instructions on how to setup the device."

Re:

[vsage3]

Not really. Far more work went into designing the setup so that it properly worked on an NES rather than an emulator, which, as TFA explains, was not an easy task because of the way lag is handled, among other things. Your suggestion would just add a constant delay to all the moves to account for the mechanical motion required.

On a related note, some of my friends thought IBM's Watson wasn't very impressive because it was receiving text input and parsing the results instead of "listening" to the answer

Re:Not all that impressive, really.

[petermgreen]

One of the key things that makes this work is that on the nes the game controls polling of the controller and the controller polling process involves controlling a shift register in in the controller. Afaict his board locks to one of the signals that controls the shift register and uses it to stay in sync with the game. This allows it to play through an entire game with frame accurate timing. Something that would be MUCH harder to do by mechanically actuating a real controller even if you took a sync source from the console.

And doing it without a sync source would be basically impossible.

was not an easy task because of the way lag is handled

Lag probablly isn't a particaully good term for the issue. AIUI the issue is that there is not a 1:1 mapping between controller input cycles and frames out output. Since his device is locked to controller input signals rather than dispaly frames he needs to strip out the frames that don't have a corresponding input cycles.

Re:

[PKFC]

And I had a dream to build a mechanical setup to control a 360 controller to "TAS" Trials HD. That game begs to be demonstrated as a perfect play. But alas, I don't have any skills like that so the dream is dead.. I suppose it would be easier to try to take apart a controller, learn the signalling and basically have a minute long macro button. Any takers?

Re:

[IorDMUX]

As a long-time fan of tasvideos.org, I have to say that I, at least, am quite impressed by the very first transition of a movie to real, physical hardware... even if they had to get the controller out of the way.

It is not so much a seminal technical achievement, as a true landmark for TAS fans and aficionados.

Re:

[KDR_11k]

Once it's perfected it'd settle all disputes of emulation error vs real bug.

Welcome Slashdotters

[Anonymous Coward]

Welcome Slashdotters.

An introduction [tasvideos.org] might be in order. Please note we have many other movies [tasvideos.org] for other consoles. We also have a list of recommended movies [tasvideos.org] for first-time viewers.

Please don't beat up our poor server too badly.

Re:

[Shikaku]

No it's not. Well, the effects are a bug, but it's an oversight. If you do the long crouch jump repeatedly, backwards, you can gain as much velocity as you want. If you do it though you may glitch through walls, which this takes advantage of, heavily.

Re:

[KDR_11k]

As in Big Rigs the movement speed limit isn't checked for backwards movement (there's also other things that get broken by backwards movement like the infinite staircase).

Re:Welcome Slashdotters

[damn_registrars]

Please don't beat up our poor server too badly.

Worry not. After the latest "upgrade" here traffic at slashdot is down enough that we couldn't take your server down even mid-day during the week when geeks are avoiding work. On the weekend when former slashdot readers are reading other websites hoping for relevant news, you have nothing to worry about.

Re:

[ashpool7]

yes, that's funny, but if you're serious, have you tried turning on low bandwidth, simple design, and D1? All I noticed when Slashdot switched was that it looked less cluttered.

Machine replaces man

[Anonymous Coward]

Great. All it needs to do now is develop an ego and look down on people who just want to have fun with their games, and this device can replace every speedrunner on the internet. Another job lost to machines...

Re:

[randomforumposter178]

it totally does take more time, and yes, that is tool-assisted walking.

Re:

[martin-boundary]

Question: if I take a bus, does that count as tool-assisted walking?

Well, where would you put it? The average pocket is too small, and anyway lugging all that heavy metal around can't be good for your spine.

Re:

[randomforumposter178]

Do they really? I think speedruns are great, but I don't think that doing that is in any way superior to just playing a game to have fun. In fact, it's really just having fun by abusing the game mechanics to their absolute limits. In fact, when you play a game normally and when you sequence break and glitch your way across the finish line in record time, it's a totally different experience. Different things stick out and different things matter. This is true even if you do your speedruns live.

Sprite collision modified?

[Anonymous Coward]

Sure looks like he's going through the flesh-eating flowers and the rotating balls to me, or is the game just not accurate enough to detect these collisions?

Re:

[Pawnn]

Actually, the game's accurate enough that if you pay very close attention, he's just BARELY dodging those things. Within pixels.

Hitboxes

[tepples]

Sure looks like he's going through the flesh-eating flowers and the rotating balls to me

Hitboxes for most enemies in SMB1 are only half as tall as small Mario.

Re:

[JonySuede]

go there for more details :
http://tasvideos.org/GameResources/NES/SuperMarioBros.html#WalljumpSmb1Smb2j [tasvideos.org]

Re:

[noidentity]

It's all legit. The device behaves like nothing more than a human who can press and release the controller buttons in arbitrary patterns every frame. The kind of abuse of glitches is a whole other world with tool-assistance.

Re:

[KDR_11k]

There's a lot of funny things in the Mario physics that you wouldn't notice when playing the game just using what you see and what the manual says. E.g. the logic for deciding whether you jumped on an enemy or got hit by it is "are you moving downward?", as long as it's met you can touch the enemy anywhere and it counts as jumping on it, even if you hit its foot with your nose. I used that to jump on the giant spiders hanging from the roof in Paper Mario, it doesn't matter that they're above you as long as

Invincible?

[Anonymous Coward]

Mario appears to be invincible here.

Re:

[petermgreen]

but some of it involves doing crazy things like pressing up *and* down on the controller at the same time.

Is that even possible on the original physical controllers?

Re:

[toddestan]

Or find one of the third-party controllers that used the Playstation-style 4 buttons instead of the D-pad.

Re:

[KDR_11k]

The Playstation's "buttons" are connected to a single disc inside the controller's case.

Holy bug exploitation

[damn_registrars]

That run exploits bugs from SMB that I didn't even know where there. I counted at least 4 times that he ran through walls in one way or another; to say nothing of the way that he would jump on the edge of a pipe while the flower was out and not take a hit.

Re:

[demonlapin]

My favorite is when he jumps against walls in 8-2 (start at 3:50) to get out of the pit.

And does anyone know why he's usually jumping facing backward? Does Mario go farther that way?

Re:

[Anonymous Coward]

Strangely enough Mario does accelerate faster this way. Don't know why. Maybe to allow players to do emergency moves in midair?

Re:Holy bug exploitation

[Soilworker]

http://themushroomkingdom.net/bugs/smb [themushroomkingdom.net]

everything is there.

Re:

[dissy]

And does anyone know why he's usually jumping facing backward? Does Mario go farther that way?

It helps with the collision detection with enemies, walking through walls, and to jump off the side of bricks, among other things.

Here is a list of pretty much all the exploits used, with animated GIFs and explanations, from the same site containing the recorded key press files:
http://tasvideos.org/GameResources/NES/SuperMarioBros.html [tasvideos.org]

Re:

[randomforumposter178]

That was my favorite too. I wonder if that's where they got wall-jumping and triangle jumping from.

Re:

[Mprx]

Totally legit, but you need frame accurate timing to pull it off. Most of these glitches are impossible for unaided humans to do.

Re:

[PRMan]

My Youth Pastor loved Mega Man (the original NES version) and would routinely do about half the glitches you see in the TAS. He used to solve the game in like 12 minutes with all the cheats.

Re:

[blackraven14250]

I've seen people do the flowers trick before a few times. Collision detection is much different now than it used to be.

Re:

[tepples]

the C64 had pixel-perfect (taking transparent pixels into account) hardware collision detection.

Which would be useful for a game like SMB1 or The Great Giana Sisters, not so much for a game using a semi-overhead view like Zelda. In this case, you want to do collision based not on the objects' projection onto the screen but on the objects' projection onto the plane of movement. (Some games represent this projection as a shadow.)

Re:

[A Life in Hell]

Yes, but the c64 can't actually use that, for the most part, if you're also using a sprite multiplexer, which eliminates its use from almost every game since there is only eight hardware sprites and no hardware multiplexer like on the nes and friends, since the engineers assumed you could just do that in software (it's documented as such in the PRG, for example).

So most c64 games also suffer hitboxes.

Re:

[blackraven14250]

Not really that it's worse now than it used to be. Now we have pixel perfect collision detection available in every 2d game due to processing power increases, or polygon-perfect in the case of 3d games.

Re:

[KDR_11k]

It just turns out that you don't WANT pixel perfect collision. A part of it is that the sprites represent 3D objects and a pixel collision doesn't say that those pixels actually look like they should be touching because they seem to have different depths.

Re:

[damn_registrars]

I don't know who pissed in your wheaties, but it may be worthwhile to note that indeed typing "where" when you mean "were" is not that uncommon, and does not say anything about the intelligence of the person at the keyboard.

Re:

[Shikaku]

SPEEDrun

Re:

[KDR_11k]

Speed is the only factor that can be objectively measured to declare one thing superior to another. With TASes a run is considered better if it's faster, unless it fails on entertainment value. If a fast run breaks the game so badly that it's unrecognizable there's a category split where a second type of run is defined that is not allowed to do specific things. A problem is when those things cannot be defined well. Some games don't even have hard limitations on the way they're played, a chasm that you're su

Re:

[izomiac]

If you're fine with exploiting glitches in a game, just enter debug mode to jump to the end "You Win" sequence for a 2 second speedrun for games with such a feature. The entire point, IMHO, is to demonstrate the skill of the player, not the ability to cheat. That's like receiving a trophy by physically stealing it rather than winning an event, it's missing the point of the accomplishment.

Re:

[Shikaku]

There [youtube.com] are [youtube.com] runs [youtube.com] like [youtube.com] that. [youtube.com]

But usually they have a non-FUBARed version to go along with it. I recommend watching at least the first and last, for the biggest WTF factor.

But most games DON'T have giant bugs like this. Most of them also would look really close to the human speedrun version, if the glitches are not too egregious to do. A Japanese person did do a 0 star speedrun under 10 minutes on a real console with no programmed controllers (i.e he used his hands and a controller alone).

Re:

[ildon]

There are plenty of non-glitch speed runs. They're just A LOT LONGER and usually (but not always) not as fun to watch.

Re:

[damn_registrars]

There are plenty of non-glitch speed runs. They're just A LOT LONGER and usually (but not always) not as fun to watch.

That is a matter of opinion. Some of us prefer to see the game done without exploiting bugs and glitches, to see what is possible within the intended possibilities of the game.

Re:

[ildon]

Some of these games take hours to complete, even with super human reflexes and perfect execution, but without exploiting glitches, bugs, and sequence breaks. I don't really know anyone willing to sit and watch that, as opposed to the 5-20 minutes many of the glitched/TAS speed runs take of the same games.

Re:

[ildon]

The skill being displayed is not playing the game, but a combination of creative thinking to figure out how to shave time off the run and coding the script to implement the idea. If that doesn't interest you, don't watch them. Like I said there are plenty of "normal" speed run videos to be found.

Re:

[Dr Herbert West]

That's not a bug, or human-impossible-- it's the way the hit boxes are set up. Similar to doing a "pit" maneuver on the spiky-wheeled cars in the original "Spy Hunter", I used to do that all the time back in the day.

Re:

[damn_registrars]

That's not a bug

Did you watch the video? He runs through solid walls more than once, and climbs out of a bottomless pit like some kind of ninja. Those are bugs for sure.

Re:

[QuantumBeep]

SMB has walljumps. They are extremely hard to do.

Re:

[toddestan]

The wall jump thing is almost certainly a bug. Though unlike things like going through solid walls which shouldn't happen, the wall jump thing can be thought of as more of an unintended feature.

Re:

[Dr Herbert West]

RTFV? You must be new here ;)

My post only referred to the "standing on the edge of the pipe" trick, I should have been more clear. Sorry for the confusion.

That set aside, I consider that maneuvers that a human can perform only with extreme difficulty (frame-accurate timing, hitting multiple buttons at once) to be exploits, not bugs. Splitting hairs, I know.

Re:

[KDR_11k]

Yeah but running through walls is how the minus world got discovered so it's not impossible in realtime.

Re:

[ildon]

to say nothing of the way that he would jump on the edge of a pipe while the flower was out and not take a hit.

It's funny that people keep mentioning that as impressive, as it's something I used to do when I was like 8 years old playing this game.

This won't work with all games

[Dwedit]

This won't work with all games. Many games made by Konami use aggressive random number generators which depend on the number of clock cycles consumed while the game waits for the next frame.

Then the game also plays digital samples through the DMC channel, and the DMC channel does not have a known consistent power-on state. Because the DMC channel periodically interrupts the CPU to fetch bytes, that will affect the number of cycles it takes while it waits for the next frame.

So we have two issues coming together, number of leftover cycles from power-on state, and initial DMC state, so when you play one of these games, the game's own demo mode isn't even consistent between multiple power-ons.

You can try it with Blades of Steel or Double Dribble, and see that their demo modes aren't always identical between different power-ons, even when you are aren't touching the joypad.

No chance of those games being TASed on hardware.

Other games are far more friendly, and don't rely on exact timing for their random number generator behavior.

Re:

[QuantumBeep]

Yeah, I do have a problem with someone calling bullshit out of stupid animal reflex. Thirty seconds of reading the articles or the above and below comments would have explained that no, hitboxes are actually pretty forgiving in SMB if you exploit them just right.

When I was 10

[CookieForYou]

When I was 10, I sunk probably a thousand hours into that game and never beat it. I got to the last Bowser a few times and died and wanted to cry.

It sure is fun to see it beat in 5 minutes.

Ok, what?

[MattGWU]

3:54, about to fall down a hole and does Mario Galaxy-style parkour to get out of it. You could do that in original Super Mario Brothers?

Re:

[DeHackEd]

Not officially, but collision detection can be abused to trick the game into believing a solid wall is ground beneath Mario's feet, allowing him to jump. Frame-precision is needed, hence the tool-assisted aspect of it.

The tasvideos.org page has more info in their wiki.

Re:

[splerdu]

You can do that by hand. I could back in the day, but only for some places like when I just barely miss a ledge and walljump back up.
The trick is to hit Mario's foot on the 16th pixel of the block, and jump immediately after touching it.

It's kinda the same for passing through (breakable) walls, except this time you hit mario's head on the upper block to cause his body to get bumped into the wall.

It's easy as big mario, try for yourself on 4-2 near the start where there's a huge corridor with breakable block

Next up: HFT!

[martin-boundary]

Next up, High Frequency Tetris on sub millisecond collocated hardware!!

Re:

[hitmark]

Are the blocks there random? If so one would need a screen reader of some kind.

Human players

[Anonymous Coward]

Though I am impressed that someone got an Arduino to do something like this, I still think that it's impressive that it only barely squeaked by the "real" record. Here's a human player (not Tool-assisted):
http://www.youtube.com/watch?v=bRyhpxR3l_g

By my count (seconds are "game-seconds"):
1-1 Tie
2-1 Arduino up 1
4-1 Tie
4-2 Arduino up 3
8-1 Tie
8-2 Human up 4
8-3 Arduino up 2
8-4 Arduino up 2

So if my reckoning is correct, the Arduino, with all its glitches, only beats a human player by 4 in-game seconds. Though SDA counts time from when the games starts to finishes, including loading times between stages, in which case it looks like the Arduino may have won by 5 real seconds. Still, the human players have done incredibly well on this game to fine-tune it to the point they have. Yes, I am a geek.

That's nothing...

[Anonymous Coward]

You should see how many updated movies on TASVideos (which I've been visiting for a long time now) beat each other by a few *frames* (i.e. 1/60ths of a second).

It's not uncommon to have a movie obsoleted in favor of a 2-3 frame improvement, especially in the Mega Man games, which are very, very, very competitively TASed.

Re:

[Anonymous Coward]

The 2-3 frame improvements generally only occur with games where the previous run was "perfect". Like Super Mario Brothers.

Have you seen the latest Mega Man 1 TAS??? The guys skip most of BombMan and the first two Dr Wily stages using a buffer overflow and a race condition...it saved three minutes over the previous best run.

That's right, a buffer overflow and a race condition. In a FUCKING CONSOLE GAME!

Uninitialized memory for randomness

[Anonymous Coward]

Uninitialized memory is not random, and relying on it to produce randomness is very bad.
Interestingly I once saw this in some C code:

static int c; /* leave uninitialized to make as random as possible */

Re:

[Dwedit]

That would go in the .bss section, and on most platforms, is automatically initialized to zero by the crt0 code. The ctr0 code is the real entry point of your program, and it's what calls main().

Re:

[KDR_11k]

These are consoles with no clock, there is no source of randomness beyond the user's controller input so developers look for anything they can find to add entropy. IIRC the PRNG in the Mega Man games is based on the number of sprites drawn on a frame in which the screen has scrolled.

TAS?

[lyinhart]

I've never found tool-assisted speedruns entertaining at all. Legit speedrun videos themselves are kind of boring, but at least they're for real. It's no different than going through a game with invincibility, or infinite time. You're changing the rules of the game - like giving yourself as many mulligans as you want in golf. Or editing out all the frames in which you missed a couple of pins in bowling to make it look like you got a 300. And the worst part is that sometime videos aren't marked as tool-assis

Re:

[LambdaWolf]

I agree that the videos themselves are rather boring to watch, except in some instances when they're paired with fun music or interesting commentary, and that people need to make it clear that they're tool-assisted. (As long as I'm griping, I also hate the totally unnecessary neologism "speedrun" when the phrase "speed run" would have sufficed, and even that is annoyingly redundant. [End semantics Nazi rant.])

But I think it's weird to suggest that they're somehow illegitimate, just because they run on diffe