Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Neural Net Learns Breakout By Watching It On Screen, Then Beats Humans

Soulskill posted about 8 months ago | from the but-can-it-eat-doritos-all-day? dept.

AI 138

KentuckyFC writes "A curious thing about video games is that computers have never been very good at playing them like humans by simply looking at a monitor and judging actions accordingly. Sure, they're pretty good if they have direct access to the program itself, but 'hand-to-eye-co-ordination' has never been their thing. Now our superiority in this area is coming to an end. A team of AI specialists in London have created a neural network that learns to play games simply by looking at the RGB output from the console. They've tested it successfully on a number of games from the legendary Atari 2600 system from the 1980s. The method is relatively straightforward. To simplify the visual part of the problem, the system down-samples the Atari's 128-colour, 210x160 pixel image to create an 84x84 grayscale version. Then it simply practices repeatedly to learn what to do. That's time-consuming, but fairly simple since at any instant in time during a game, a player can choose from a finite set actions that the game allows: move to the left, move to the right, fire and so on. So the task for any player — human or otherwise — is to choose an action at each point in the game that maximizes the eventual score. The researchers say that after learning Atari classics such as Breakout and Pong, the neural net can then thrash expert human players. However, the neural net still struggles to match average human performance in games such as Seaquest, Q*bert and, most importantly, Space Invaders. So there's hope for us yet... just not for very much longer."

cancel ×

138 comments

Sorry! There are no comments related to the filter you selected.

I for one (4, Funny)

fisted (2295862) | about 8 months ago | (#45796219)

I for one welcome our new virtual ass-kicking overlords.

Re:I for one (0, Troll)

Anonymous Coward | about 8 months ago | (#45796257)

Hurr hurr. You as funnay as Dane Cook and Carlos Mencia combined!!

Re:I for one (0)

Anonymous Coward | about 8 months ago | (#45796281)

Well, based on their highs in popularity, maybe he should go on tour?

Re:I for one (-1)

Anonymous Coward | about 8 months ago | (#45796367)

You never fail to find at least one retard who keeps on using that line over and over again wherever applicable. The internet community scoffs at scolding them because in their eyes that's about the same as scolding an honest to goodness, medically backed, retard for acting stupid.

Re:I for one (0)

Anonymous Coward | about 8 months ago | (#45796559)

Oh No! Not scoffed at by the Internet Community! Oh No! The Horror! Oh wise plugged in-one, what is an acceptable course of action?

Re:I for one (1)

Anonymous Coward | about 8 months ago | (#45797925)

This is the weakest form of trolling I have ever seen. I award you no mod points and may God have mercy on your soul.

Excerpt from "Starfish" by Peter Watts (5, Interesting)

Anonymous Coward | about 8 months ago | (#45796229)

"I hope the lifter pilot doesn't get too bored." Jarvis is all chummy again.
"There is no pilot. It's a smart gel."
"Really? You don't say." Jarvis frowns. "Those are scary things, those gels. You know one suffocated a bunch of people in London a while back?"
Yes, Joel's about to say, but Jarvis is back in spew mode. "No shit. It was running the subway system over there, perfect operational record, and then one day it just forgets to crank up the ventilators when it's supposed to. Train slides into station fifteen meters underground, everybody gets out, no air, boom."
Joel's heard this before. The punchline's got something to do with a broken clock, if he remembers it right.
"These things teach themselves from experience, right?," Jarvis continues. "So everyone just assumed it had learned to cue the ventilators on something obvious. Body heat, motion, CO2 levels, you know. Turns out instead it was watching a clock on the wall. Train arrival correlated with a predictable subset of patterns on the digital display, so it started the fans whenever it saw one of those patterns."
"Yeah. That's right." Joel shakes his head. "And vandals had smashed the clock, or something."
"Hey. You did hear about it."
"Jarvis, that story's ten years old if it's a day. That was way back when they were starting out with these things. Those gels have been debugged from the molecules up since then."
"Yeah? What makes you so sure?"
"Because a gel's been running the lifter for the better part of a year now, and it's had plenty of opportunity to fuck up. It hasn't."
"So you like these things?"
"Fuck no," Joel says, thinking about Ray Stericker. Thinking about himself. "I'd like 'em a lot better if they did screw up sometimes, you know?"
"Well, I don't like 'em or trust 'em. You've got to wonder what they're up to."

Can we get a summary of that excerpt, please? (-1)

Anonymous Coward | about 8 months ago | (#45796447)

Maybe it's just because it's out of context, but that excerpt is extremely difficult to follow. It could just be the writing style, too. Sci-fi often employs a very simplistic, nondescript form of writing that's difficult to comprehend when not part of the full work.

In any case, could somebody who can understand it please summarize what idea or ideas that excerpt is trying to convey, in comprehensible English? It probably won't take more than a sentence or two, I believe. A lot of that writing appears to be filler and fluff, but it's difficult to tell where exactly that begins and ends.

Re:Can we get a summary of that excerpt, please? (0)

CODiNE (27417) | about 8 months ago | (#45796545)

Summary of post:
tl;dr

Re:Can we get a summary of that excerpt, please? (2, Insightful)

Anonymous Coward | about 8 months ago | (#45797039)

tl;dr == "I hate reading". You should NEVER see a tl;dr at slashdot. NERDS READ.

Re:Can we get a summary of that excerpt, please? (0)

Anonymous Coward | about 8 months ago | (#45797111)

Nonsense. Just because something is in written form it does not mean that there's value in reading it. This is especially true if it's longer, yet inherently worthless, written material that'll waste a lot of time and effort to read through. The excerpt is a good example of this. It's rambling, obtuse, and pretty much incomprehensible. One can read it, but there's nothing of value to be obtained by reading it. Another good example is your comment. Yes, it involves some reading, but its lack of substance and insight, if not its outright incorrectness, means that reading it is a pointless activity.

Define Slashdot.org (1)

narcc (412956) | about 8 months ago | (#45799225)

It's rambling, obtuse, and pretty much incomprehensible. One can read it, but there's nothing of value to be obtained by reading it. [...] its lack of substance and insight, if not its outright incorrectness, means that reading it is a pointless activity.

Re:Can we get a summary of that excerpt, please? (4, Informative)

themightythor (673485) | about 8 months ago | (#45796547)

In this case, the "gels" were employing a heuristic to know when to do something (in this case, turn on the air ventilation system). It was assumed that it was something meaningful to the action (i.e. something to do with the recipients of the ventilation), but it was something arbitrary (i.e. the way the clock looked). So, unless you have insight into what the heuristic is, you won't know when it's going to have the expected behavior and when it isn't. Even if it seemingly has the expected behavior for a long time.

Re:Can we get a summary of that excerpt, please? (1)

mhajicek (1582795) | about 8 months ago | (#45796933)

The performance would be better than that of a human regardless. A human will get high or drunk and smash the trains into each other once in a while.

Re:Can we get a summary of that excerpt, please? (1)

Anonymous Coward | about 8 months ago | (#45796573)

Jarvis and Joel are discussing smart gel - some kind of AI that apparently has had bugs. What was difficult to understand?

Re:Can we get a summary of that excerpt, please? (0)

tomhath (637240) | about 8 months ago | (#45796993)

Aside from the weird writing style, the whole excerpt doesn't really make sense. They assume because the software hasn't killed anyone yet that it's perfect. Not a good assumption. Then he says he would like the software more if it did kill them.

Re:Can we get a summary of that excerpt, please? (1, Insightful)

ElectricTurtle (1171201) | about 8 months ago | (#45797689)

"Hey, I have an idea, let's take concepts, deliberately misunderstand and exaggerate them, and then the person who created the concepts will look stupid!

Oh, wait, that's a dumb idea, because we'll end up looking like the stupid ones."

That is the conversation you should have had with yourself before you posted.

In the excerpt one of the chars expresses a begrudged acceptance of the 'gels' because they haven't 'fucked up' which is not, despite the anecdote which precedes the opinion, exclusive to fatalities. The responding party understands this, because he's not a total idiot, and says that he wishes the 'gels' made some kind of mistakes (again, with NO exclusivity to fatalities as you ridiculously assert in your summation).

Make me wonder how people like those in these comments ever passed verbal standardized tests. Reading comprehension is negligible and it seems even actively avoided.

Re:Can we get a summary of that excerpt, please? (1)

rmstar (114746) | about 8 months ago | (#45798799)

Aside from the weird writing style, the whole excerpt doesn't really make sense.

Alternatively, your mind is just too rigid to dig the style and too inflexible to get what the dialog is about.

Re:Can we get a summary of that excerpt, please? (0)

Anonymous Coward | about 8 months ago | (#45796587)

Must be you, I understood it easily. It's a conversation between two people, Jarvis and Joel, about the dangers of smart gels (gel form neural network), based on an incident a decade ago.

Re:Can we get a summary of that excerpt, please? (0)

Anonymous Coward | about 8 months ago | (#45796775)

You're not alone. Although I understood it, that is some really shitty writing. Really bad. I couldn't imagine reading a whole book with that writing style, it's terrible.

Re:Can we get a summary of that excerpt, please? (1)

Bucc5062 (856482) | about 8 months ago | (#45796779)

Don't trust computers if your name is Jarvis.

In the greater context I think the post is trying to make the same thought, AI's that use visuals to make decisions may not be trusted with human life.

Re:Can we get a summary of that excerpt, please? (0)

Anonymous Coward | about 8 months ago | (#45797507)

You should learn to read.

Re:Can we get a summary of that excerpt, please? (-1)

Anonymous Coward | about 8 months ago | (#45798017)

I get it. You're not a reader. Why ask for a summary [that you won't read] when you can simply watch the scene it comes from on youtube? [youtu.be]

captcha: entrap

Re:Excerpt from "Starfish" by Peter Watts (1)

Bob the Super Hamste (1152367) | about 8 months ago | (#45799367)

Reminds me of a story my AI prof told about creating an expert system for deciding when cheese was fully ripened.

The cheese company had a person who's job it was to check the cheese and would go and poke the wheels to see if they were ready. The company created a robot to go and do the same job and trained it by having it poke fully ripened and unripened cheese. The problem was that when it was put it use it failed miserably at correctly and consistently telling if the cheese was ripe. The problem was the wrong input, the old cheese tester would poke the cheese not to see how squishy it was but was poking it to let some of the smell out. I don't know how true this story was and it was told to me almost 15 years ago but it seemed reasonable.

Don't let it watch terminator (1, Insightful)

Anonymous Coward | about 8 months ago | (#45796231)

Next up wee have Sky Net.

Re:Don't let it watch terminator (1)

The Grim Reefer (1162755) | about 8 months ago | (#45797351)

Next up wee have Sky Net.

Slow down there AC. We need to get WOPR first.

Let the thing was Matrix (2)

roman_mir (125474) | about 8 months ago | (#45796245)

Let this Neural Net watch Matrix a few times, then turn on the sound and hear it say: "I know Kung Fu".

AI (5, Insightful)

ledow (319597) | about 8 months ago | (#45796253)

For once, something based on proper AI (rather than human-generated heuristics).

However - notice it's limitations: Where there is a direct correlation between where you need to be, and where something else is on the screen (basically a 1:1 relationship in Pong, for example), it can cope with going higher or lower as required.

But when you put it into something that has more than a single thing to "learn" (move left/right, avoid bombs, shoot aliens, choose which aliens to shoot, don't shoot your own base, etc.) then the amount of training required goes up exponentially. And thus we could spend centuries of computer time in order to get something that can do as well as a simple heuristic designed by someone who knows the game (not saying heuristics don't have their place!).

"Trained" devices require training relative to some power of the variety of the inputs and the directness of their correlation to the game-arena. And thus, proper AI is really stymied when it comes to learning complex tasks.

But still - this is the sort of thing we should be doing. If it takes an infant two years with the best "computer" in the universe that we know of to learn how to talk, why should we think it will take a machine at even the top-end of the supercomputer scale (which can't have as many "connections" as the average human brain) any less?

Re:AI (3, Interesting)

Anonymous Coward | about 8 months ago | (#45796337)

If it takes an infant two years with the best "computer" in the universe that we know of to learn how to talk, why should we think it will take a machine at even the top-end of the supercomputer scale (which can't have as many "connections" as the average human brain) any less?

Because we're learning languages in the wrong way.

Re:AI (2, Interesting)

GTRacer (234395) | about 8 months ago | (#45796945)

So, Pimsleur or Rosetta?

Re:AI (0)

Internetuser1248 (1787630) | about 8 months ago | (#45796339)

There is another aspect that limits this sort of machine learning, the need for direct positive/negative feedback. Games that don't have a score counter are pretty much unlearnable. Obviously you can base the feedback on something else like level reached but in some games, like kings quest for example, a neural network will not be able to figure out when it is doing well and when it isn't. Often a separate heuristic AI is needed just for this.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45796531)

King quest has score. All the sierra's 'quest' games has score. Solving puzzle increase the score and you can finish the game with different score depending on how well you did and how many optional puzzle you found. Maybe you should try playing them instead of bullshitting all over our interwebs.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45796693)

It's not a score machine learning can learn anything from: you get a point for a random action, and you never get more points for the same action.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45797179)

Worse, you get punished for doing an action you're supposed to do but in the wrong order. (e.g. eating a pie to stop a hunger alert.)

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45796831)

Lucas Arts games might be something AI could play. They're designed so that you're never supposed to be stuck so that you cannot complete the game even though some bugs can do that (e.g. the infamous horn bug in Monkey Island 2). Then all that needs to be done is to try using every object on every other object and going through every dialogue tree available to you (and to make it easier parts of the dialogue are never exposed to you when some choices exclude others without affecting the outcome). Clever players took advantage of this never get stuck feature at least in Monkey Island 1 because when you put objects in the pot they disappear from your inventory permanently and Guybrush refuses to put any such objects in it that you need later. Thus the set of objects you had to experiment with when solving the puzzles became smaller.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45797249)

Lucas Arts games might be something AI could play. They're designed so that you're never supposed to be stuck so that you cannot complete the game even though some bugs can do that (e.g. the infamous horn bug in Monkey Island 2). Then all that needs to be done is to try using every object on every other object and going through every dialogue tree available to you (and to make it easier parts of the dialogue are never exposed to you when some choices exclude others without affecting the outcome). Clever players took advantage of this never get stuck feature at least in Monkey Island 1 because when you put objects in the pot they disappear from your inventory permanently and Guybrush refuses to put any such objects in it that you need later. Thus the set of objects you had to experiment with when solving the puzzles became smaller.

That's not AI, it's brute-forcing. You don't use AI on a problem that can be solved easily and would run fast enough with even the most naive implementation.

You don't need AI when ten lines of imperative code would do!

Re:AI (1)

fisted (2295862) | about 8 months ago | (#45797615)

You should try Maniac Mansion some day. Plenty of ways to mess up, game-over style

Re:AI (3, Interesting)

StripedCow (776465) | about 8 months ago | (#45796401)

If it takes an infant two years with the best "computer" in the universe that we know of to learn how to talk, why should we think it will take a machine at even the top-end of the supercomputer scale (which can't have as many "connections" as the average human brain) any less?

Because neurons are much slower than transistors?

Re:AI (1)

Anonymous Coward | about 8 months ago | (#45796635)

A neuron is more like a network router with local integrated storage, packed in a density somewhat comparable to integrated circuits.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45798037)

NO. They are tubes.

Re: AI (0)

Anonymous Coward | about 8 months ago | (#45797283)

who told it what the scores mean? it didn't learn that on it's own.

Re:AI (0)

s.petry (762400) | about 8 months ago | (#45797641)

If it takes an infant two years with the best "computer" in the universe that we know of to learn how to talk, why should we think it will take a machine at even the top-end of the supercomputer scale (which can't have as many "connections" as the average human brain) any less?

While I kind of agree with the point, it absolutely doesn't take 2 years to learn how to talk. It takes a few months to learn to talk (which would include learning that sounds have meaning). Just like it doesn't take over a year to learn to walk, it takes a couple weeks. Interestingly, they learn some very complex things all at the same time.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45797911)

How is it proper AI though?

What proper AI is there about it? human can figure out himself what to do in the game.

Both of these games need only to come to the net conclusion that if the only moving piece is left from the paddle then move left and if to right then to right.

Re:AI (0)

Anonymous Coward | about 8 months ago | (#45798723)

"However - notice it's limitations"

You know how to use an endash, but can't tell it's from its? Time for some neural net learning!

It's called a "JavaScript Programmer" algorithm. (5, Funny)

Anonymous Coward | about 8 months ago | (#45796315)

This neural-net-combined-with-trial-and-error style of algorithm is typically referred to as a "JavaScript Programmer"-type algorithm in recent AI literature. (I'm being completely serious, too, in case you think this is a joke; it isn't.)

The name derives from the similarity between how these kinds of algorithms work, and how JavaScript programmers tend to work.

Both the algorithms and JavaScript programmers use a very basic, minute form of pseudo-intelligence.

This small dab of pseudo-intelligence is then used to repeatedly attempt to solve a problem, followed by an analysis of the success of the attempt.

In the case described in this article, it involves the computer trying to play the game, with the aim of winning.

In the case of the JavaScript programmer, it involves the programmer repeatedly searching through Stack Overflow, finding code to copy-and-paste, and then hoping that it works well enough to trick the customer or employer into thinking the job is done.

The summary should have probably mentioned this, but I suspect that the submitter may not be following the latest AI journals and research very closely.

Re:It's called a "JavaScript Programmer" algorithm (2, Funny)

cfulton (543949) | about 8 months ago | (#45796433)

Truer words were never spoken:

the programmer repeatedly searching through Stack Overflow, finding code to copy-and-paste, and then hoping that it works well enough to trick the customer or employer into thinking the job is done."

Re:It's called a "JavaScript Programmer" algorithm (1)

cascadingstylesheet (140919) | about 8 months ago | (#45797721)

Truer words were never spoken:

the programmer repeatedly searching through Stack Overflow, finding code to copy-and-paste, and then hoping that it works well enough to trick the customer or employer into thinking the job is done."

If it really works, if the specifications are met, and if it passes testing, then the job is done.

Wisely leveraging the shared knowledge of others is a good thing to do.

Re:It's called a "JavaScript Programmer" algorithm (3, Interesting)

cascadingstylesheet (140919) | about 8 months ago | (#45797611)

This neural-net-combined-with-trial-and-error style of algorithm is typically referred to as a "JavaScript Programmer"-type algorithm in recent AI literature. (I'm being completely serious, too, in case you think this is a joke; it isn't.)

The name derives from the similarity between how these kinds of algorithms work, and how JavaScript programmers tend to work.

Funny, of course :)

But, you got me thinking. The JavaScript programmer is generally trying to affect the appearance of stuff on the screen, therefore, he looks at the stuff on the screen, and tries to affect ... the stuff on the screen. So, it makes more sense than it might.

Our new pong-playing overlords, on the other hand, if they are actually doing something important like remotely fighting wars or trying to save people or something, well, then we don't really know if they are looking at the right input, and it becomes much more important that they, and we, understand exactly how they are coming to their decisions.

Re:It's called a "JavaScript Programmer" algorithm (1)

marcosdumay (620877) | about 8 months ago | (#45798051)

In the case of the JavaScript programmer, it involves the programmer repeatedly searching through Stack Overflow, finding code to copy-and-paste, and then hoping that it works well enough to trick the customer or employer into thinking the job is done.

And now, I'm wondering if there is another way for creating DOM manipulating Javascript. I mean, I can most of times make a Linux module by reading the documentation of a device and writting code that makes it work (but for some devices, it's the Javascript way), imagine some kind of data representation and then write it down with native types, imagine a Python map, a Haskell fold, or a SQL query, write it down, and in all those cases the stuff that I imagined works (given or taken a few bugs). But I could never, in my entire life, create a piece of DOM manipulating Javascript that did what I thought it should do. I always resort to trial and error.

Re:It's called a "JavaScript Programmer" algorithm (0)

Anonymous Coward | about 8 months ago | (#45798057)

Almost every AI algorithm does this. If we knew the correct choices for every game state, the problem would be quickly automated and people would forget about it. How many people are researching new Tic-Tac-Toe AIs?

Even logical inference systems do this. Behind the scenes, backtracking/DFS/BFS tries going down different states.

I know it was tongue in cheek , but... (1)

Viol8 (599362) | about 8 months ago | (#45798557)

"used to repeatedly attempt to solve a problem, followed by an analysis of the success of the attempt."

The above is exactly how humans learn to play simple games. Sure, you learn a few rules beforehand but then you actively - and to an extent subconciously - engage in trial and error about what to hit/kick/click at what time in what scenario. Its called "practice". No one for example becomes a good football (soccer for the yanks) player by analysing angles of attack of other players feet - they just go out and keep playing until they become better.

Re:It's called a "JavaScript Programmer" algorithm (1)

tonywestonuk (261622) | about 8 months ago | (#45798923)

Its also more formally called TDD. Create some code that tests the suitability of the existing code to solve the problem. Then randomly change the code until it passes the test, and all the others. Repeat, rinse, etc.

corepirate nazi spiritual bankruptcy mindphucking (-1)

Anonymous Coward | about 8 months ago | (#45796317)

leaves end users feeling almost nothing besides empty misdirected disconcert which is quickly converted into fear & hate for the unknown?

The handwriting on the wall (1)

cold fjord (826450) | about 8 months ago | (#45796335)

Deep Blue [ibm.com]

A little nit (1)

Anonymous Coward | about 8 months ago | (#45796347)

Where did the researchers find the "expert Breakout and Pong players" to match their neural net against? Was it that same loudmouth kid down the hall who is always "beating the spread" on football?

Wrong question (1)

mbone (558574) | about 8 months ago | (#45796409)

The question is not, "when can a bunch of machinery beat a human at X." The question is "when can a bunch of machinery beat a team of humans _with access to similar computational resources_ at X." I don't see much progress there.

Re:Wrong question (0)

Anonymous Coward | about 8 months ago | (#45796665)

I want to see this as a comparative currency chart. We have data about what a human can do in these games with 25 cents and the vague, faded, and torn instructions around the edge, how much does this machine have to spend to get up there with an untrained human?

Re:Wrong question (0)

Anonymous Coward | about 8 months ago | (#45797431)

"An untrained human" with 10-20 years experience at living doesn't exactly make them equal.

Re:Wrong question (1)

egcagrac0 (1410377) | about 8 months ago | (#45798825)

No problem.

I can wait 20 years for the computer to catch up.

Re:Wrong question (0)

Anonymous Coward | about 8 months ago | (#45798271)

No, the question is when can a bunch of machinery intentionally beat a team of humans with physical force sufficient to kill them? because that, my friends... is where this is heading.

12/27/2013 - machine beats human at pong.
4/14/2034 - machine beats human to death with pointy stick.
4/14/2034 - machine beats team of humans to death after it is targeted for shutdown.
4/21/2034 - machines beat army of humans in defense.
4/22/2034 - machine beats humans at evolution. Human race is now extinct.

I can already hear the daleks... (1)

crovira (10242) | about 8 months ago | (#45796431)

"Exterminate... Exterminate...."

Actually, when they become advanced enough, we won't need to work anymore.

I'll buy TWO. One to do my job and one ... just in case.

Re:I can already hear the daleks... (1)

Nyder (754090) | about 8 months ago | (#45796965)

"Exterminate... Exterminate...."

Actually, when they become advanced enough, we won't need to work anymore.

I'll buy TWO. One to do my job and one ... just in case.

Dalek's are mutate life forms, riding in a machine. They are not robots or A.I.'s.

Re:I can already hear the daleks... (1)

wonkey_monkey (2592601) | about 8 months ago | (#45797491)

Dalek's are mutate life forms, riding in a machine.

Now I have an image of the little squishy Dalek sitting inside in a little chair, turning a little wheel and going "wheeee!"

(Username recognised)

Oh, KentuckyFC (1)

Anonymous Coward | about 8 months ago | (#45796469)

You were just asking for an oblig [xkcd.org] , weren't you?

Tetris (2)

wjcofkc (964165) | about 8 months ago | (#45796557)

Tetris, by nature, would prove most interesting. I myself never made it past level 10, and I've never seen anyone make it past level 20. I wonder what the breaking point for this neural net would be after a few days of practice. I would love to see a video of it starting from level one and making it's way to the insanity of level 50 - if it's up to the task. I imagine a super computer would have too much latency.

Re:Tetris (0)

Anonymous Coward | about 8 months ago | (#45797293)

If a computer can solve Level 1, then Level 50 is not much harder --- just some linear scale up in speed. Humans have fairly slow fixed reaction times, which makes doing something a little bit faster a whole lot harder, once you're pushing up against those limits. With computers, throwing more/faster clock cycles to speed up operation is rarely the hard part of the problem; determining the algorithm to solve Level 1 (via explicit instructions or machine learning) is the hard part.

Re:Tetris (1)

wjcofkc (964165) | about 8 months ago | (#45797485)

Still, there must be an upper limit to what it can handle, I would be curious to see what that limit is.

Re:Tetris (4, Informative)

Sigma 7 (266129) | about 8 months ago | (#45797335)

Tetris is a solved problem if you're going for survival (assuming you don't get an extremely unlucky piece selection). Since AI has access to the current piece, the next piece, and can do a probability check on the next piece, it can basically last forever.

I myself never made it past level 10, and I've never seen anyone make it past level 20.

Tetris: The Grand Master: http://www.youtube.com/watch?v=jwC544Z37qo [youtube.com] - fast forward to 3:00 to see first majoor speedup, 4:45 for final speedup, and 5:01 for invisible pieces.

That, and 999999 was done on a real NES within 3 minutes 11 seconds: http://www.youtube.com/watch?v=bR0BKCHJ48s [youtube.com]

Re:Tetris (0)

Anonymous Coward | about 8 months ago | (#45799293)

Tetris is a solved problem if you're going for survival (assuming you don't get an extremely unlucky piece selection). Since AI has access to the current piece, the next piece, and can do a probability check on the next piece, it can basically last forever.

You don't even need to know what the next piece is if you're not heavily emphasizing getting blocks of 4 rows at a time. Meanwhile, getting more than 4 rows total is a major accomplishment with BasTet.

Re:Tetris (0)

Anonymous Coward | about 8 months ago | (#45797433)

If you like Tetris, you may enjoy this demonstration of skill:

Invisible Tetris [youtube.com]

Re:Tetris (1)

wjcofkc (964165) | about 8 months ago | (#45797525)

I would like to see a thorough scan of that guys brain compared to an average brain. It appears he is only looking at the next piece while remembering everything that came before it and their positions.

Re:Tetris (0)

Anonymous Coward | about 8 months ago | (#45797759)

He probably visualizes the layout of the pieces much like a chess master - able to fully comprehend the meaning of a board layout at any given instant.

Many good chess players can play blindfolded, remembering every move that has gone before. However, I don't think many of they could make 200-400 moves per minute!

His motor skills must be extraordinary. He is somehow combining a total knowledge of an extremely rapidly changing playing field with phenomenally fast, well-practiced hand movements - he surely has no time for any conscious thought at those speeds.

Huh? (0)

Anonymous Coward | about 8 months ago | (#45796567)

"Now our superiority in this area is coming to an end."

You mean our superiority in BREAKOUT and PONG. This hardly applies to EVERYTHING.

Play itself (1)

Anonymous Coward | about 8 months ago | (#45796627)

They should spin up two instances of the neural net and have it play itself

All is lost! (2)

portwojc (201398) | about 8 months ago | (#45796629)

The AI has another advantage over us human players with the Atari 2600. No blisters.

Re:All is lost! (1)

GTRacer (234395) | about 8 months ago | (#45796975)

Also, it isn't tempted to pull the rubber cover off the stick and suction it to its forehead. Do you know how many forehead hickeys I gave myself with that stupid thing?!

But (2)

Stargoat (658863) | about 8 months ago | (#45796677)

But has it learned to let someone else design Breakout and then steal a couple thousand dollars from him for his efforts? When it does that, it will truly be an intelligence. (And it will be a superior intelligence if it leaves off the black turtlenecks.)

Minecraft (1)

Anonymous Coward | about 8 months ago | (#45796685)

Show me an AI that can play minecraft; that would be impressive.

Re:Minecraft (1)

Anonymous Coward | about 8 months ago | (#45797325)

Define the goals of minecraft. If you mean "An AI that can form its own aesthetic desire of what a nice house/castle/statue/dick should look like and creates it for fun," I'll agree with you, but if the standards are low, I can make an AI that plays minecraft by looking straight down and holding left click for a few minutes. I'll call it a zombie survivalist AI.

Re:Minecraft (1)

Anonymous Coward | about 8 months ago | (#45799059)

Kill the dragon in hardcore mode.

Perhaps I can teach it (2)

The-Ixian (168184) | about 8 months ago | (#45796711)

to farm gold for me.

Separate score and such input? LSTM? (1)

speckman (2511208) | about 8 months ago | (#45796717)

I would think making sure the score section of the net gets special emphasis might help with the harder games. Separate inputs even with the numbers of various things known to a human player (score, lives), rather than having the AI get that from a bitmap and separate/extract.

Also, I'm guessing they're not using all the tricks one can with neural nets. Like long short-term memory. That would seem to seriously help with this sort of thing. Basically I'm guessing their lack of success with the harder games is not due to inherent limitations like some of the above posters said, but due to limitations in their implementation.

That's Not Impressive (1)

CanHasDIY (1672858) | about 8 months ago | (#45796751)

So it can play Breakout, big deal.

Wake me when it's giving the checkers-playing chicken a run for her money.

Space Invaders (0)

Anonymous Coward | about 8 months ago | (#45796811)

Lrrr: You are defeated. Instead of shooting where I was, you should have shot where I was going to be. Muahahahaha!

What is the missing piece (1)

Bucc5062 (856482) | about 8 months ago | (#45796847)

The interesting part of the slim article was the part left out. Why did not not perform as well on some of the games. There was not much detail on that issue. I'm not familiar with the poorly played game, but I would guess they introduce a level of visual complexity that overwhelms the AI?

Other than that, simply astounding accomplishment.

Nothing new here (0)

Anonymous Coward | about 8 months ago | (#45797109)

Remember have read about this in Scientific American some 20 years ago. It did not use neural nets but matchboxes and colored beads, but could learn tic-tac-toe.

Vajk

Dear God (0)

Anonymous Coward | about 8 months ago | (#45797155)

Don't let it play Missile Command

I for one... (1)

jd.schmidt (919212) | about 8 months ago | (#45797235)

...Welcome the new King of Kong!

wining is pointless (1)

shadowrat (1069614) | about 8 months ago | (#45797381)

Winning means nothing unless you can enjoy it. Is the computer having any fun?

Re:wining is pointless (2)

egcagrac0 (1410377) | about 8 months ago | (#45799437)

It's learning to have fun.

Run for the hills! (1)

wonkey_monkey (2592601) | about 8 months ago | (#45797429)

Neural Net Learns Breakout By Watching It On Screen, Then Beats Humans

Women and children and nerds first!! The machines are coming!

Oh, my mistake. I thought it said "neural learns to break out" and then something about beating humans.

This is one reason why most people don't Capitalise Every Word In A Headline.

Already Done (0)

Anonymous Coward | about 8 months ago | (#45797891)

People would learn more, faster if they spent more time reading. A University of Alberta student published a very similar thesis in 2010: http://www.arcadelearningenvironment.org/wp-content/uploads/2012/07/Naddaf_2010_Game-Independent-AI-Agents-for-Playing-Atari-2600-Console-Games.pdf The work is so similar I had to double-check that the authors weren't the same and simply continuing their work at a different university. The thesis tested various reinforcement learning methods as well as search-based solutions. The basic idea was the same. Graphically watch the games, then learn to replay them. I think the thesis did a better job then these guys.

I really hate how researchers claim they're always first to do some new novel thing. The only sort-of new thing they did was try out the same problem with a different learning algorithm. If someone has already shown it's possible with one algorithm, it's going to be possible with another. I'd have more respect for them if they didn't say they were the first. I see no reason to include such statements in research papers except for ego stroking.

**not an AI advancement** (0)

globaljustin (574257) | about 8 months ago | (#45798437)

TFA does not describe an advancement in AI technology whatsoever.

It is an external 'computer player'...We have had AI's that play video games virtually since we had video games.

Take good ol' Tecmo Bowl...you play against an AI opponent that does absolutely everything this AI did and more.

This is not an AI advancement, it is....an **application** of new and better **sensor inputs** for an external AI

I don't see anything in this that would indicate we are some kind of 'step' closer to having Terminator kill bots....it's just an application of visual pattern recognition to a particular task.

Another example of this tech being in use today is assembly line robots. They are programed to behave according to certain visual parameters input from visual sensors.

This is **application** of existing technology...engineering...not new science or AI evolution....this is HYPE

Neural Net Also Sings (1)

srobert (4099) | about 8 months ago | (#45798721)

"Would you like to hear the song I learned today while we play? Daisy, Daisy, give me your answer do ..."

Waiting for computer to get so smart..... (1)

3seas (184403) | about 8 months ago | (#45798943)

... they don';t need us creators any longer. Now if only we could see what they come up with about their origins after we are gone.

The 2600 is from the '70s (0)

Anonymous Coward | about 8 months ago | (#45798949)

Nuff said.

Comes with a caveat (1)

Salgat (1098063) | about 8 months ago | (#45799067)

This is neat but it needs to be understood that mimicking what other players do without the understanding of strategy and deeper conceptual thought severely limits what the AI can do. This AI could never learn to play sophisticated games simply because it works by copy-pasting basic behavior. Even something with as basic rules as Go would be far beyond this AI.

More AI Hyperbole (1)

mlookaba (2802163) | about 8 months ago | (#45799551)

Very regularly, someone writes a clever new algorithm to crunch a specific limited set of data more efficiently.

Repeat it with me: "This is not an AI breakthrough".
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>