Google One Step Closer to Replacing Humans

DeepMind Finally Dominates Starcraft

In a sad day for humanity, Google’s DeepMind finally dominates world class Starcraft players.

For the 80% of readers whose first response is, “What’s Starcraft?”, allow me to explain the significance of this event:

• In 1997, Kasparov lost to Deep Blue (Chess)
• It then took almost 20 years for Google’s AlphaGo to defeat world Go Champion Lee Sedol in May 2016
• Real time strategy games held out for a little longer, since they added several layers of complexity – including processing in real time and non-discrete “moves”. However, various AI programs made rapid progress, including OpenAI’s defeat of world class DOTA teams in 2017.

Starcraft was one of the first highly profitable and wildly successful real time strategy (RTS) games, which basically consisted of sprites moving around a 2 dimensional map that was non-discrete (i.e. no squares or hexes). It includes multiple game concepts – an economy system, technology development, construction, military, special units, and fog of war. It is hideously complex, and some players spend a decade mastering it. At one point, it became so popular in South Korea that it was unofficially dubbed the national sport.

Due to its complexity, the DeepMind team identified five key challenges:

1. Game Theory (outguessing the opponent)
2. Imperfect information (fog of war)
3. Short vs. long term planning (tradeoffs and investments to maximize future win probability)
4. Real time processing
5. Large action space (hundreds of units/actions)

If you ever played against a computer AI in a video game from 10 years ago, it’s generally not that challenging unless the computer AI cheats (for example, gets extra units or more resources). Most “artificial intelligence” systems built by game developers could only loosely be called AI. Like most “AI” systems in current business automation or ‘intelligence’ software, it’s mostly a set of rules and logical conditions that are coded by humans, sometimes with a bit of prediction or classification thrown in. The Guardian called these “Wizard of Oz techniques”. Once you figure out those rules, it’s not that difficult to “hack” the AI and win the game.

DeepMind is different… It’s the real deal. Google’s team built a nice visualization of the AI “processing” decisions during gameplay (below). If you want to see a visualization of DeepMind (“AlphaStar”) processing, check it out here.

Google’s team spared no effort in developing AlphaStar. They developed a custom structure that mapped out the features and action set of the game, essentially enabling it to take key actions – focus attention, interpret events, map events to an evolving ontology which supports an open ended “understanding” of those events, and then apply actions which sequentially improve the probability of a victory. They also did NOT CHEAT. They limited their attention space and actions per minute (“APM”) to a median range for a world elite player. (Some of the world’s best players routinely execute 400 actions per minute, or over 6 actions per second…)

So what does this mean?

There are three really important concepts to take away from this event.

1. AI developers are getting vastly better at mapping complex decision spaces onto modular AI domain structures. AI developers are getting much better at thinking of AIs as modules or components – much like the human brain relies on a modular structure. This modularity – and, eventually, recurring patterns and strategies to develop modules – is going to become the standard for real AI development. This will enable some AI developers to specialize exclusively on some components (i.e. image or voice recognition), and others to specialize on using those modules in higher order programs.
2. DeepMind’s developers made a big point of incorporating “multi-agent” learning. This is just a fancy name for combining evolutionary approaches with AI scoring, something that researchers have been doing for a couple decades (starting with the Santa Fe Institute). In fact, the evolution of artificial life became the focus of another highly awarded game called Species. The kicker here is that breeding AI agents and simulating games in parallel allows DeepMind to train itself thousands of times faster than a human.
3. Once an AI program is built, it can be replicated… literally copied without requiring retraining. The elite world competitors are all unique, with their strengths and weaknesses. People get old, get tired, and get sloppy. They make mouse-click errors and get frustrated. Not the AI.

Google’s DeepMind researchers are already (and with good reason) considering a wide range of applications for the tool sets they successfully proved out. Military applications are certainly high up on that list, even if Google itself isn’t working on them. The applications are simply too obvious. More on that later.