Why artificial intelligence is taught to play video games

What motivates you to continue a video game? A simple explanation is curiosity. As it turned out, it is key effective motivator for learning artificial intelligence playing video games, writes The Verge.

The study OpenAI explains how AI by the curiosity surpassed his predecessors, who played in the classic game of 1984, “Montezuma’s Revenge”. Victory in this game, of course, not equal to victory in the Go or Dota 2, but still quite significant. The report DeepMind 2015, it was reported that the AI, after a few Atari games and using deep learning, in “Montezuma’s Revenge” scored no points.

The reason for the difficulty is the mismatch between gameplay and learning methods of artificial intelligence. Agents commonly rely on AI reinforcement learning in the development of video games: they are “immersed” in the virtual world, getting rewards for certain achievements (e.g., glasses) or Vice versa, they are punished (loss of life). Thus, the agent learns to play by trial and error. Reinforcement learning is often viewed as a key method to create more intelligent robots.

Зачем искусственный интеллект учат играть в видеоигры

The problem with “Montezuma’s Revenge” is that the game has no regular remuneration for the agent AI. This is a puzzle-platformer where players must explore an underground pyramid, avoid traps and enemies, collecting keys that open doors and special items.

If you have trained the agent AI to win the game, you could reward him for what he survived and collected the keys, but how do you teach him to maintain certain keys to certain items and use them to overcome enemies and complete the level?

The answer is simple: curiosity.

The study OpenAI agent was rewarded not only for jumping over pits with spikes, but also for exploring new parts in the pyramid. This has led to improved performance, and bot has received an average score 10 000 (in comparison with the average human score of 4 000).

“There is still a lot of work. But what we have at the moment is a system that can explore a lot of rooms to get benefits and sometimes to get through the first level. The levels are similar, so the walkthrough of the whole game – just a matter of time,” said Harrison Edwards of OpenAI.

Зачем искусственный интеллект учат играть в видеоигры

The fight against “noise TV”

The researchers used the concept of curiosity as a motivation for decades. Interest-based forecasts are only useful when training for certain types of games such as Super Mario.

Another problem is the “noise TV”, where agents AI, programmed to seek new experiences, “addicted” to random patterns, such as static noise TV. Agents perceive the “new and interesting” as what to do with their ability to predict the future. Before AI will take a certain action, it will predict how the game will look then. If the guess is correct, he’s probably already seen this part of the game. This mechanism is known as “error of prediction”.

But since static noise is unpredictable, any agent AI, faced with such a TV, it becomes hypnotized. OpenAI compares the problem with people addicted to slot machines – they can’t because they don’t know what will happen next.

Зачем искусственный интеллект учат играть в видеоигры

OpenAI researchers bypassed the problem by changing how the AI predicts the future. Precise methodology, Random Network Distillation, complex, Edwards and his colleague Yuri Burda comparing this to hide the secrets of AI. The mystery is random and meaningless – something like “what color is in the upper left corner of the screen?”, but it motivates the agent to explore, protecting him from the traps of “noise TV”.

More importantly, this motivator does not require a large number of calculations. Such methods of reinforcement learning are based on huge amounts of data. “The method they use is actually quite simple and, therefore, unexpectedly effective,” commented software engineer, Unity Arthur Giuliani. – “Given the similarity between different levels in the “Revenge of Montezuma” OpenAI, in fact, equivalent to solving the game. But the fact that the AI still can not get through the first level, means that there were certain issues”.

Зачем искусственный интеллект учат играть в видеоигры

The importance of curiosity

What is the benefit of curious artificial intelligence? Curiosity helps computers to learn on their own. Most approaches to machine learning today can be divided into two parts: first, machine learning, learning data by developing templates that they can apply to similar problems; secondly, they are “immersed” in the right environment and getting rewards for certain achievements using reinforcement learning.

Both approaches are effective in specific tasks, but also require a large amount of human labor. AI providing an essential incentive to research, people spend less time for training.


Share Button