Computer program Learn FUN: Learns to play NES games

The ending makes it.

Dumbo

This is scary mind blowing.

Pft

No need to break out the EMPs yet

<><

But think of it this way: It figured out the concept of Rage Quit all on its own.

We can panic when it can beat Battletoads. On one continue.

Of course, don't you know anything about ALCHEMY?!- Twin clones of Ivan the Great

Hopefully a computer program that can learn like this won't go Maverick.

Please help out our The History Of Video Games page.

Set it on I Wanna Be The Guy and it'll destroy mankind out of sheer frustration.

Dopants: He meant what he said and he said what he meant, a Ninety is faithful 100%.

Okay this is really cool.

That... actually looks kinda impressive, and rather clever. It may not do well on all games, but I do wonder whether it might not be still improved such that it makes better decisions.

A weighting mechanism might get it around the Tetris issue, for example, as long as one had a way to weight the negative result of stacking bricks (perhaps by looking into how Tetris checks for a stack passing the top line, or by looking at the memory that represents the playing field and weighting against whatever corresponds to adding a block above the top-most, or giving higher weight to placing blocks lower, or some such)...

edited 13th Apr '13 1:26:32 PM by ArsThaumaturgis

My Games & Writing

The whole point of the program is that it's not individually tailored to any particular game. You could certainly make it better at Tetris by telling it what to care about, but that would also defeat the point entirely. He just feeds it some sample gameplay to give it some data to start from and the program decides on its own which values it wants to raise and what inputs it needs to best raise them.

The problem with Tetris is simply that it doesn't look enough steps ahead. Obviously achieving a Tetris (or even just clearing a line) is worth more points than planting a block on top of another, but it can't see any way to reach one of those endpoints from where it is within the scope of its foresight so it chooses the much shorter term benefit.

edited 13th Apr '13 1:59:46 PM by Clarste

Hmm, you're right, I do believe — I'd forgotten or missed the point that, beyond initial training, it shouldn't call for game-specific modifications.

Perhaps then some built-in method of detecting paths that are not game-ending, but which are potentially less desirable than others? I'll confess that I'm no great player of Tetris, so I'm not sure of how good players think about a given level, and I an example from another game isn't occurring to me, so I don't have something from which to attempt to extract an algorithm offhand.

\*shrugs* But then perhaps you're correct in saying that all that's called for is allowing it to look yet farther ahead.

edited 13th Apr '13 2:13:25 PM by ArsThaumaturgis

My Games & Writing

This is pretty cool.

I've done enough programming to have some familiarity with how computers can come up with totally bizarre, yet sometimes relatively effective, solutions to problems they are presented with. For instance, did you know that for any integer x from 0 to 14, x^x (that is, x to the power of itself) expressed in binary has a 1 in at least one of the '2' or '4' positions if and only if x is prime? No? Neither did I, until my C program figured it out.

Thinking about the Tetris example in particular, I suspect that a better way to improve its performance would be to allow it to construct its own heuristics. That is to say, instead of only trying to improve actual in-game memory values, let it compute new values based on those in memory and learn which of those are associated with raising the values in memory. This might let it determine, for instance, that packing a high density of Tetris blocks at the bottom of the screen is desirable. It might also be less resource-intensive than simply having a deeper search tree.

Join my forum game!

It seems to me that the main problem is rather that it doesn't seem to be aware that stacking blocks has any negative side-effect, only that it gets points for doing so. In other words, it doesn't see the somewhat distant failure state.

Allowing it to construct its own heuristics might help if you allow it to train itself. It might then be able to learn that the failure state comes into play when the memory associated with the stacks reaches this point, and thus that raising stacks is undesirable.

My Games & Writing

Video has a sequel!

Yeah, I know I'm necro'ing this thread, but I did so because I think I have something worthwhile to add.