StarCraft AI Competition Results 113
bgweber writes "The StarCraft AI Competition announced last year has come to a conclusion. The competition received 28 bot submissions from universities and teams all over the world. The winner of the competition was UC Berkeley's submission, which executed a novel mutalisk micromanagement strategy. During the conference, a man versus machine exhibition match was held between the top ranking bot and a former World Cyber Games competitor. While the expert player was capable of defeating the best bot, less experienced players were not as successful. Complete results, bot releases, and replays are available at the competition website."
I for one (Score:3, Funny)
am terrified we're actively pursuing the ability for our robot overlords to have the perfect micromanagement strategies.
Re: (Score:2)
Re:I for one (Score:5, Funny)
Re: (Score:1)
As far as I know, there are no deployed technologies that shoot [missles] or drop bombs without human intervention.
Re: (Score:1)
Served! Served! Served! Served!
Re: (Score:1)
Re:I for one (Score:4, Funny)
Yes but that's mounted on a warship, that mode can be turned off (by manually disconnecting the weapons if necessary, there are people on board after all) and that mode would only be turned on if a shooting war broke out and they needed to make sure that the ship wasn't going to get blown up before they could hit the "shoot down that motherfucking missile screaming in at us" button.
It all adds up to something a bit different than an autonomous weapons platform that can choose to engage targets at will, which is what other people were talking about.
Re: (Score:1)
Not equal (Score:5, Insightful)
By default, I'd say that a well-designed bot in an RTS would have an advantage over all but the best players. Since there is so much to keep track of, software would win out over wetware.
Bots in an FPS are one thing, but when you have dozens of units, a build order, multiple fronts, resources, and more to track all at the same time, the infinitely scalable multitasking of a bot would certainly come in handy.
Or maybe I'm off base?
Re: (Score:2, Insightful)
It doesn't matter if you have the best multitasking in the world if the opponent can just outright kill you cause it has more stuff.
Re: (Score:2)
Re: (Score:3, Interesting)
Re: (Score:3, Interesting)
The point is, it's really easy to just outsmart an RTS AI once you figure out its tendencies. No matter how good it's multitasking is your own won't be bad enough that your flying units all of a sudden forget how to shoot down.
Having an intricate knowledge of counters and economy and timings(at least as far as starcraft goes) has always been superior to multitasking, but only to a point.
Re: (Score:3, Insightful)
Re: (Score:2)
This is true, but it doesn't matter how good your micro manage is if the units aren't in the right place. Simple drop strategies and resource starving are going to beat these types of bots.
Re: (Score:1)
Re: (Score:1, Informative)
you're not, and it's interesting that these Ai's only have as much knowledge of that game as a normal player would, no magic ESP or extra resources to make up for in efficiency. My favourite one was an AI that could play the game for you, but you as a human would give it overarching decisions, I couldnt find it on a quick google but there's a video of someone playing on youtube and I reckon it'd beat me hands-down.
Re:Not equal (Score:5, Informative)
It's a huge advantage but it's only one part of the game. Many would argue that overall strategy is far more important than micro-management.
One of the top protoss players (WhiteRa) in SC2 (former SC1 player) has a relatively low actions-per-minute (APM) count compared to most players. Yet he still comes out on top by a lot.
Being able to multitask and micro-manage is definitely an advantage but a far more important ability is being able to plan on the larger scale. I've never seen an AI capable of harassment techniques, guerrilla warfare and exercising map control. Multi-pronged attacks are also something that it should theoretically be great at but it never really tries.
Re: (Score:2)
Re: (Score:2)
My experience has been that SC2 matchmaking is pretty reasonable, and there are a TON of totally bad players out there to give n00bs like you and me a reasonable win percentage. So yeah, you'll be ranked super-low but your win percentage will probably be okay.
Re: (Score:3, Interesting)
Re: (Score:2)
Mostly because if standard AI did this, players would raise hell. No human can really keep up with 2 100% controlled and coordinated attacks. Mainly because we have to split our resources between them. The AI could realistically perfectly control 2 attacks with all the skill that a human player could devote to a single attack.
Even with all the hot keys in the world you still need to flip your view bet
Re:Not equal (Score:4, Interesting)
> No human can really keep up with 2 100% controlled and coordinated attacks. Mainly because we have to split our resources between them. The AI could realistically perfectly control 2 attacks with all the skill that a human player could devote to a single attack.
The biggest problem is the INTERFACE of the game. Let me know when I can create on-the-fly Picture-In-Picture overviews of the map in real-time, so I _actually_ can attack/defend on multiple fronts.
Sad to see RTSs really haven't changed in 20 years ;-(
Re:Not equal (Score:4, Interesting)
An interesting modification to StarCraft which would give AIs a run for their money would be cooperative play where several users operate one team on the field, and where the several users have a hierarchy and delegate command of corps of units to other players, maybe one player handling resources while another scouts while a third consolidates the offensive force.
Re: (Score:3, Informative)
Not "would" - does. That feature has been in the original StarCraft ever since. The original Age of Empires had that feature, too.
It's sad that they didn't include this in SC2 though.
Re: (Score:2)
Re: (Score:1)
Yes this is definitely the best feature of original SC! I loved how the team of players could also choose different races. And you started with the command structure of player #1 (Human CC for example), but you got your own drone or probe as a starting worker to build a Nexus or Hatchery later!
Personally I love coop in a RTS! Another game that had this feature was Conquest: Frontier Wars. Excellent game too. I recall one match where we had 3 players vs 2 players, with each side controlling only one empire.
Re: (Score:1)
Re: (Score:3, Informative)
This is basically what we would do in 4 v 4 Age of Empires. The person farthest from the other team would be "resource bitch". They would focu
Re: (Score:2)
Rrrrrreally. That sounds pretty cool, though I'd probably be as terrible at it as I am at Starcraft.
Re: (Score:2)
The great equalizer is that the computer would be leading half of his army to each attack, which would lead to annihilation of one of his two groups. Depending on what damage to the ba
Re: (Score:2)
That's hardly an equalizer. The player would also lose a chunk of his forces in battle and there is no guarantee that the player is keeping 100% of his forces grouped in one spot. The point is with 2 completely controlled attacks, it's quite likely they could easily attack your resources and wipe that out while keeping you busy somewhere else.
Re: (Score:2)
A lot of it is imho putting the human player on guard, playing mind games and making the other player think that you have more than you actually etc. something that most bots are inherently immune to. Intelligence and demoralization beats brute force most of the time in human warfare. Especially in electronic war games you can easily sacrifice a couple (or even a whole lot) of units in order to take the majority of the army out of position.
Look at Iraq for an example, there was bunches of brute force, the U
Re: (Score:2)
Re:Not equal (Score:5, Interesting)
Bots in FPS ARE a completely different thing, as setting up an auto-headshot sniper that will hit the player before he can get the bot into view is not that difficult to make. That field will never be an even battle.
Strategy games are a little different though. A Bot essentially has no "better" way to evaluate the player than any other player would evaluate the player. Say in Starcraft, the Bot scouts the player - and determines he is a little behind in what he would expect the player's army count to be. This could mean a number of things: The player made a mistake, the player is saving up, or theres something the player has that you have not found yet. How do you proceed?
Now - when you get to the pro level of gaming, you worry a little less about your opponent's build and worry more about not letting them know yours. Walling and other defensive techniques become just as important as scouting your opponent. The game becomes highly a higly reactive scenario as opposed to proactive. If you know what your opponent is doing, you can counter it and that puts you much further ahead, possibly ahead enough to crush them.
So the problem eventually lies in getting an AI to properly counter a players actions. Making an AI react to players is much harder than giving an AI a plan and telling him to execute. Because essentially the reaction is only as smart as whoever is programming the AI. And if you are a better player, capable of keeping other people from determining your plan, you can beat an AI who is trying to determine yours.
Don't get me wrong, the ability for computers to instantly Micro and Macromanage all of the units and resources at once does give it some serious advantages, but deep in the heart of it: The AI will only be a little better of a player than the person who programs him. (Or her, if you program female AI's like GLaDOS)
Re: (Score:3, Informative)
as part of the rules for this contest the AI had to have fog of war turned on.
Re: (Score:2)
IIRC In SC2 the Insane AI doesn't have a fog of war, but the other levels all do. All of the other difficulties play a fair game.
Re:Not equal (Score:5, Funny)
You will be Zergrushed. And then there will be cake.
Re: (Score:2)
Bots still tend to outplay in that they have instant communication amongst other bots (Like the voice chat) and have quicker reaction times.
Even "on Equal Footing" Bots tend to play better, the only time they don't is when you purposefully detriment their regular abilities like accuracy and detection. Essentially if you and a bot were walking near each other, just a grenade lob away, the bot would be able to instantly know you are there because the instant the sound is registered in their detection range th
Re: (Score:2, Insightful)
Re: (Score:3, Insightful)
Re: (Score:1)
Starcraft is only rock-paper-scissors if you don't scout.
It's still RPS even if you do scout, the only difference being that if you scout, you know the player will choose rock, so you know to choose paper.
Essentially the, it's rock-paper-scissors-lizard-spock-zergling-zealot-marine-mutalisk-archon-[...]
Re: (Score:2)
Re: (Score:2)
Uh, you scout and realize they have paper so you build scissors. What are you talking about?
Re: (Score:2)
This would be true for initial battles against that AI, but at some point, the human player will come to understand the AI's tricks, which will always be finite. Even if the AI is sophisticated enough to change strategies when faced with possible defeat, it would be entirely possible for a human player to learn how to stay "on the threshold" so-to-speak and simply overrun the AI at the right time.
Knowing the AI's strategic plan is ultimately more important than being able to micromanage with more efficienc
AI strategy (Score:2)
I've seen a lot of AIs that have a general strategy that they stick to like glue. I'd love to see an AI that continually evaluates its current position, and if it sees a loss trend, switches tactics.
When you're doing the right thing, but slowly bleeding out to inevitable death, you might want to lower your risk aversion and try something else.
Beyond that, some way of evaluating a player and keeping track over multiple games - and comparing known players and their styles to new players to make predictions,
Bots lack creativity (playbook-limited) (Score:1)
While a bot will certainly be able to macro and micro better than a human, human players are closing the gap and I don't think that the current gap is all that much between humans and machines.
On the other hand, the creativity gap between humans and machines is always in the favor of humans. At best (as of now), machines can only be designed to react, and their "creativity" is limited to a book of plays. Once you know the machine's playbook, you can easily defeat them. So they are only as good as how well
Re: (Score:2)
I disagree. Starcraft 2 has a lot of different strategies to account for. Sure, the bot should do a better job of being able to micromanage, build efficiently, not screw up on food, etc but a decent human will beat it because it will struggle to account for tricks that a human can come up with. A good human will implement its strategy on the fly. They will scout, see what the bot has and develop a counter. It has a big army? Do some drops in the back of their base. And usually a bot will have a flaw..
Re: (Score:1)
On the other hand, the SC2 AI is great if it should always on the offensive. My team has been hit with coordinated attacks that prevent us helping each other, especially when they run away.
Re: (Score:1)
Re: (Score:2)
Your idea is correct, but the use of the word infinite is off-base. That's like saying chess is infinitely variable. It's simply not true. There's only a small number of openers in both games. And from any situation, there's only a small set of viable next moves (somewhat true in chess - especially true in SC where the game flow depends heavily on what happened before).
In the beginning two minutes of the game, if you have more than 40 APM you're just doing something excessive. Even through midgame, until yo
Important note (Score:5, Informative)
Uses Broodwar, not Starcraft 2, and not just the original.
Just saying. (Cause when they mentioned Ultralisk Microing, I thought about SC2 and how Ultralisks are terrible units there simply because they block your units making Micro a huge pain, and it wasn't so bad in BW when your units could take a bit more of a beating).
Re: (Score:2)
And you probably can count how many times he did after 1.1.1 patch on one hand =P
Re: (Score:2)
FruitDealer dominated his games from start to end, ultras were an icing on the cake. The truth is they offer much less bang for the buck than they used to in sc1. They have splash damage, but they are so huge now and that makes them a pain to micro effectively. They don't really synergize with other zerg stuff. Remember ultraling+dark swarm?
Re: (Score:2)
After a while the AI in sc1 is dead simple to beat. It likes to send in units into your base at the same spot one at a time. Just place some static defenses there, and you can macro without being disturbed, and then when you're maxed run over it.
Now the sc2 AI is much better, I still have troubles coping with it in certain situations (esp early rushes) starting at the "Very Hard" difficulty. It doesn't wall off its base though, so sending in 50 speedlings can kill its economy early on, which causes it to ne
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Re: (Score:1)
Re: (Score:2)
That's the best strategy? (Score:4, Interesting)
Re: (Score:2, Interesting)
Against Muta's? Just spam Goliaths.
Re:That's the best strategy? (Score:5, Informative)
Re:That's the best strategy? (Score:4, Informative)
It's actually pretty interesting technique that they used. They don't just clump up the mutas. From the results page:
Contemporary StarCraft wisdom tells us that the best way to use mutalisks is to clump them. In human versus human battles, this makes it difficult to single out the weaker mutalisks, because the units are stacked on top of each other. However, UC Berkeley’s team identified a flaw in this tactic; it reduces the damage output of each individual mutalisk, because not all mutalisks will fire when using this tactic. Instead, they employed a model in which mutalisk are always moving, maximizing damage output while simultaneously maximizing movement.
Video can be found at the bottom of the page [ucsc.edu].
Re: (Score:3, Informative)
Any 'learning' bots? (Score:4, Interesting)
Anyone know if anyone has ever created a bot that has the ability to learn from losses and wins, to figure out what works and what doesn't (both what it is doing and what the enemy is doing, then use that in the future to predict what the other player is *trying* to do and come up with a counter)?
I've not played a lot of RTS's, but I've played a few, and the thing I've noticed is, if a strategy works against a bot, even if there's a fairly obvious counter, it will always work against that bot.
The RTS I've played the most is Supreme Commander: Forged Alliance, so I'll use an example from that: There are a collection of different bots available which will use different strategies. Now, the player can build a strategic missile launcher, which builds nukes. You can also build a strategic missile defense which will build 'seeker' missiles which will shoot down incoming nukes.
Some of the bots will build SMD's, some won't, but in no case does it build an SMD based upon the player/opponent's actions. That is, it doesn't check to see if you have started to build any Strategic Missile Launchers before it begins to build the defenses. The bots that don't build defenses won't build them even if you are building one. This means on the one hand that the bot wastes resources which it could have used otherwise, to build defenses, while on the other hand, if it doesn't build them, you can pretty easily and quickly defeat the bot with a nuke or two. Alternatively, instead of building a strategic missile defense, the bot *could* try to use someattack method (for example, if you don't have good air defenses, it could hypothetically try a targeted attack with a bunch of bombers or gunships to either destroy the SML, or the engineer units which are constructing the SML).
However, the bots never seem to be smart enough to attack the obvious threat of a strategic missile launcher. It seems like the only way the developers found to make the bots harder is to make them much more efficient at building up their economy and spamming out lots of land-units to try to attack the player.
When I get a chance, I want to try SC2, but right now, I'm in a period where I'm not playing games as much as I used to, and trying to reduce my gaming down to almost none while I get some more important things done in RL. It'll be interesting to see how the AI differs in that game.
Re:Any 'learning' bots? (Score:4, Funny)
Re: (Score:1)
Re: (Score:1)
Re: (Score:1)
Are there bot v. bot competitions in SC? (Score:2)
Re: (Score:1)
In any case, I'm happy that bots and their authors are getting some prizes and recognition, because I think their work is incredibly important and fertile. Bots are out future.
I, for one, welcome our new bot overlords.
Re: (Score:1)
Nerd Fantasy Extrodinaire: Ingame Scripting Agent (Score:5, Insightful)
Competitions like these are great, but they (and games like SCII) really make me pine for an RTS with a robust in-game scripting language. I would like to write complex auto-executing instructions for my units like "Pursue but never enter the firing zone of a known enemy turret or siege tank" or "If your energy > X and enemy of type Y in range R, cast spell S" or "If you are unit type X, always position yourself between friendly unit type Y and the enemy". You could confine the script to some reasonable specifications (say, no more than 1000 queries and 100 orders given per second) if you want to deter brute-force approaches.
This is a totally different problem than writing a good AI, you would be focused on writing powerful tools that aid, not replace, a human player by letting him specify his intent on a higher level than "go here" or "attack that". Better visualization of what's happening would be an integral part of this too. I would love to have SCII give me an overview of what units/building I have, what they are doing/queued to do -- even better if they are grouped into functional 'squadrons'. Being able to have multiple panes/monitors looking at different things would help to.
I guess what I'm saying is that I'm old and slow and want to leverage my experience writing rule-based logic to beat the whippersnappers that can click faster than I can and keep more things going in their heads at once :-P.
Re: (Score:2)
Taken to the logical extreme, it sounds like you just want a game where you don't have to micro your units.
Actually, I think the extreme is a game where you write the script then you don't manage your units at all -- and it turns out that games like that are pretty awesome [sourceforge.net] even when you only have one unit to manage, so I can only imagine managing a whole army would be much more fun (presuming the complexity wasn't overwhelming)
Re: (Score:2)
hmm what if you had to code during the game (no way to save/restore programs), and you're also limited there by a resource? So you can't implement that 5000 lines of code AI, because you only have 10 seconds for it, and you only have collected enough energy for 5 lines of code.
Re: (Score:2)
hmm what if you had to code during the game (no way to save/restore programs), and you're also limited there by a resource? So you can't implement that 5000 lines of code AI, because you only have 10 seconds for it, and you only have collected enough energy for 5 lines of code.
Yeah that sounds sweet, you could also just make it so the more complex the units code is the longer it would take to train. You could also unlock new functions as you progress, and have a spy unit to steal enemies various code.
Re: (Score:1)
Re:Nerd Fantasy Extrodinaire: Ingame Scripting Age (Score:5, Interesting)
Re: (Score:2)
I kinda liked the "less control" approch of Majesty.
Your units actually think for themselves, and respond differently to your "reward" incentives. To me this makes for a more "realistic" experince than the "hive mind" approach of the standard RTS (Where the player controls exactly what every unit does.)
Re: (Score:1)
Re: (Score:1)
I'm glad to see a potential player base for this type of RTS. I would also love the ability to select from scripted build orders. Just let me fine tune bits of my second-nature strategies in game and let me concentrate on scouting and such.
Re: (Score:2)
I think this is an awesome idea. I would love to play a game like supreme commander but options like that. It would turn every game into a real battle of strategies, instead of who can best micromanage their resources.
Re: (Score:1)
That was me BTW.
For instance, imagine you're given a set of parts with simulated physics and physical+logical interconnects. .... we could have competitions to see who can program the fastest running bot!
I figure with real robots there would be a lot of welding and metal-piece-shaping involved, but imagine if somehow all that was virtual/programmable, and the whole thing moved according to your programming (lower leg goes up to xx degrees, has such and such torque/force/movement curve).
BOT DANCE COMPETITION
Re: (Score:2)
That was me BTW.
For instance, imagine you're given a set of parts with simulated physics and physical+logical interconnects. .... we could have competitions to see who can program the fastest running bot!
I figure with real robots there would be a lot of welding and metal-piece-shaping involved, but imagine if somehow all that was virtual/programmable, and the whole thing moved according to your programming (lower leg goes up to xx degrees, has such and such torque/force/movement curve).
BOT DANCE COMPETITIONS!
AMERICA'S NEXT TOP ROBOT!
Hmmm... I may sound like I own a Wii :), but IMHO you don't want to make it too complex, just a simple macro based system, or perhaps an event based flow chart. There also needs to be some sort of penalty for overly complex orders.
Several non-obvious flaws (Score:5, Informative)
1) In the flying units versus flying units match, mutalisks (guys with wings) should have focused the scourges (little 'c' shaped guys) because scourges have about 1/5th the health of a mutalisk, but can suicide to take out 5/6ths of a mutalisk's health. Red ai focused scourges, the other ai didn't, with disastrous results for the other ai.
2) In the match with infantry, the players had medics, which heal other units but can't attack, and marines/ghosts, which can do damage but can't heal each other. One ai moved medics with ghosts such that the medics could actually heal. The other ai just left the medics a mile away from the combat, and got slaughtered. Furthermore, the AIs didn't bother with formations, which meant that half their units spent the entire fight trying to get into range. A precombat formation lets almost all the combat units start firing as soon as the fight begins.
3) The red zealots retreated in the face of numerically superior opponents, while the teal zealots just attack moved no matter how many they had. Teal zealots didn't focus fire, which meant that they lose units sooner, and thus had less damage output compared to red. In addition, red failed to kill the pylons (600hp) powering the buildings (a lot more than 600hp). Neither player built their bases to maximize the number of pylons powering their vital gateways; each pylon usually powered only one building.
4) In the fight with dragoons (orange spider things) versus tanks, the protoss (orange) could have frozen half the enemy tanks with a single stasis spell by sneaking the arbiter (flying spider thing) to the back of the tank formation. Furthermore, the protoss could have focused the science vessel (floating teal circle) that was preventing them from being invisible. It would have been a slaughter if the vessel had been focused, as teal would have had no real way to hit the orange units.
5) In the match between Overmind and Krasi0, the article talks about mutalisk clumping preventing some of the mutalisks from attacking. However, the point of the stack is that when one guy is in range, everyone else is. Also, the attack animation is so fast that for all practical purposes, the flyers can shoot while moving. They use their mobility to get out of range of infantry units, then zoom in to pick off outliers when their attack cooldown is finisihed. In actual competition, the terran player would usually rely on a strategic placement of static defenses and a highly mobile cluster. However, the terran (defending force) built tanks, vultures, and goliaths (mech guys that shot missiles) with the flaw that tanks and vultures can't shoot air, and are thus almost useless versus the mutalisks. The mutalisks spent the second half of the clip shooting tanks rather than focusing down the goliaths volleying missiles into them. The terran AI prioritized repairing tanks as much as goliaths, and didn't place tanks next to goliaths to soak up bounce damage from a mutalisk.
In general, the AI had problem with understanding the priority in a fight. That is to say, they often had no sense of what units are critical to a position or what units pose the most threat. They didn't arrange their units to maximize their effectiveness, and often failed to alter their behavior based on the other party. It's a fun contest, but I'm not sure the AIs could beat a moderately skilled player who understands tactics AND strategy.
Re: (Score:1)
Generic VS Specific (Score:3, Interesting)
What I would be curious about would be a more generic AI. Instead of using known s
haha (Score:1)
UC Overmind (Score:3, Interesting)
Here's an overview of our strategy. Broadly, our agent always plays Zerg, and its primary objective is to apply constant pressure to our opponents, which will let the agent continually expand to improve its economy. To that end, our agent had three primary tools: a scouting worker that harassed early, zerglings that provided early defense, and mutalisks which basically force the opponent to stay in their base. Our agent would make decisions, based on what it observed, to trade off between these different forces. It might build more zerglings if it sensed more early pressure, or it might skip them altogether if our opponent wasn't going to attack early.
Our choice of mutalisks was deliberate. They are a highly mobile all-purpose air unit that can mass up and cause significant damage in a short period of time. Also, they don't have bounding boxes, meaning that they're more amenable to computer control. That choice proved to be really valuable, because a lot of the opponents seemed to have preferred ground armies. However, there are many other strong units. For example, the other agent in the finals (Krasi0) was truly impressive with its ability to repair units.
Anyway, on our page we have a couple of videos, and we'll have several more by tomorrow (Saturday).