DeepMind's AI is learning to play soccer
"To understand the football, we must actually solve many problems on the road to general artificial intelligence [Agi] - explains Guy Lever, researcher at DeepMind -. There is to consider the control of the entire humanoid body, coordination, a really difficult thing for the 'Agi, mastery of low-level motor control, long-term planning ".
The experiment Artificial intelligence must recreate everything human players do on the pitch - including unconscious actions, such as precise movements of the limbs and muscles needed to make contact with a moving ball - making hundreds of decisions per second. The timing and control required for even the most basic movements can be surprisingly difficult to recreate, as anyone who has ever played the Qwop browser game will remember. "We humans do it without thinking about it, but all of this is a really difficult problem for artificial intelligence, because we ourselves don't know how we do it," says Lever.
DeepMind simulated humanoid agents were modeled on real humans, with 56 points of articulation and a limited range of motion. This means that they are unable, for example, to rotate the knee joint in ways that seem impossible like Zlatan Ibrahimović can. To begin with, the researchers simply assigned virtual players a goal - such as running or kicking a ball - letting them try to figure out how to accomplish them by trial and error and through reinforcement learning, as was done in the past when researchers taught virtual versions of humanoids moving between obstacle courses (with comical and rather unnatural results).
Content This content can also be viewed on the site it originates from.
"It didn't work," says Nicolas Heess, another DeepMind researcher and co-author of the research with Lever. Due to the complexity of the problem, the huge range of options available and the lack of prior knowledge, virtual footballers had no idea where to start, just squirming on the ground. Heess, Lever and other colleagues decided to resort to neural probabilistic motor primitives (Npmp), a teaching method that pushed the artificial intelligence model towards more human-like movement patterns, in the hope that the basic knowledge would help solve the problem of how to move on a virtual soccer field. "In practice, motor control is oriented towards realistic human behavior - explains Lever -, which is learned through motion capture applied to humans playing football".
In this way, space is reconfigured action ", continues Lever. The movements of the humanoids are already constrained by the body and by the joints which can only bend in certain ways; exposure to data provided by real humans is an additional limitation, helping to simplify the problem. "You are more likely to discover useful things by trial and error," Lever points out. Npmps accelerate the learning process. A "subtle balance" needs to be found between teaching AI to do things like humans and giving them the freedom to find solutions to problems on their own, which may prove more effective than we come up with. br>
Training The basic training was followed by training reminiscent of those of a real football player: running, dribbling and kicking the ball, imitating the way in which humans could learn to practice a new sport before try your hand at a game with other players. Reinforcement learning taught simulations to chase a target without the ball or dribble. The acquisition of these skills is a natural way to perform increasingly complex tasks.
The aim was to encourage virtual players to use skills learned outside the sporting context within a football environment, in order to learn to be flexible in switching from one movement strategy to another. Subsequently, those who had learned to master the techniques were employed as teachers. In the same way that she was encouraged to mimic what she had learned from human motion detection, AI was also rewarded when she chose not to stray too far from the strategies used by teachers in particular scenarios. "This is actually a parameter of the algorithm that is optimized during training - says Lever -. Over time, in principle, they can reduce their dependence on teachers".
The first games After training, it's time to play a game: the researchers started with 2v2 and 3v3 to optimize the experience accumulated during each simulation round. The highlights, which you can watch below, convey the same feeling of chaos as a dog chasing a ball in the park: rather than running, the players stumble and are perpetually on the verge of falling to the ground. Goals are not scored thanks to elaborate passes, but to fortuitous actions.
Content This content can also be viewed on the site it originates from.
Although in matches AI players were only rewarded for scoring goals, the researchers quickly saw characteristics such as teamwork emerge: "At the beginning of the training all the players were running towards the ball, but at some point, after a few days, we saw that when they became I realized that one of their teammates was in control of the ball and they started to turn around and go up the field, predicting that the teammate would try to score or pass the ball, "reveals Lever. It is the first time that such complex and reactive artificial intelligence demonstrates such coordination and teamwork skills: "This is one of the most interesting discoveries for me," continues Lever.
Applications for the robotics The goal is not to win the robotics world cup; Heess is working to transfer some of the basic AI skills into physical robots so that they can move more "safely and naturally" in the real world, and not just to avoid frightening humans interacting with them. The erratic, jittery movements that can result from an unstructured learning process could harm robots that haven't been optimized to move that way, or simply waste energy.
This is all part of the work on the so-called embodied intelligence, the idea that general artificial intelligence may have to move around the world in physical form, and that it is the nature of this form that determines how it behaves. "It's interesting both for simulated worlds, which increasingly feature physics-based simulations, and for developing methods for learning robots," Heess notes.
Ultimately, these uncoordinated digital gamers could help both robots that metaverse avatars move in ways that seem more human, even though they'll never beat us in football. "Football is not a final goal in itself - underlines Lever -. There are only a lot of things to solve before getting to that point."
This article originally appeared on sportsgaming.win US.
