Scientists invent algorithm that can make AI learn continuously

Researchers from Google’s DeepMind unit and London’s Imperial College have developed an algorithm that allows artificial intelligence to learn continuously. This should solve the problem that such systems ‘forget’ previously performed tasks.

The researchers refer to this phenomenon in their paper as ‘catastrophic forgetting’. The DeepMind project explains that this occurs when a neural network has to learn a new task. This overwrites the ‘knowledge’ that was acquired during the execution of the previous task. This creates a limitation for the learning capacity of neural networks. In the paper, the researchers present a method to overcome this limitation.

To achieve this, they developed an algorithm they call ‘Elastic Weight Consolidation’. The idea behind it is that the algorithm indicates for each connection between different ‘neurons’ how important it is for performing a particular task. When learning a new task, this value protects the old connection from modification. The degree of protection depends on the importance assigned to the old task. The researchers draw the comparison with a spring, which becomes stiffer the more important the task is.

The impetus for the research was the functioning of the human brain and the way it handles learning new tasks. In addition, the process scientists refer to as systems consolidation is responsible for transferring memories from the part of the brain that learns quickly to the part that learns more slowly. This process is influenced by the conscious and unconscious recall of memories, the scientists said. In addition, there is a process called synaptic consolidation, in which connections between neurons are not overwritten if they were important in performing a previous task.

To test their algorithm, the researchers conducted experiments on Atari 2600 games. For example, the AI ​​agent played several games in succession. It turned out that an agent without the algorithm quickly ‘forgot’ a learned game. However, using the algorithm, the system was able to learn multiple games in succession without losing the knowledge gained in the other game. The scientists want to use this to demonstrate that ‘catastrophic forgetting’ is not an insurmountable barrier and that systems can be designed that can learn efficiently and flexibly.

Other research projects also use games to train machine learning systems. For example, in addition to Atari 2600 games, OpenAI also uses Red Alert 2 and Portal to develop a system that can use a computer just like a human being. In addition, the organization uses GTA V for training self-driving systems.