Bio-Inspired Deep Reinforcement Learning for Autonomous Navigation of Artificial Agents

Abstract

Autonomous navigation of artificial agents is a challenging task for changing and complex environments. Reinforcement learning (RL) algorithms are widely used for autonomous navigation, where the agent, through the interaction with the environment, learns the behaviors needed to maximize the reward. Recent architectures extract information from the environment using convolutional neural networks, where the visual features needed to maximize the reward are unknown and uncertain, and then, increasing the number of parameters learned by the entire system. Moreover, the presence of sparse rewards complicates, even more, the task generating unstable results in the learning problem. The work here presented is twofold. First, we show the advantages of using retina physiology knowledge to design a visual sensor feeding the RL network. Secondly, based on intrinsic motivation, we propose the use of auxiliary tasks to deal with sparse rewards, generating a continuous learning process. We define two auxiliary tasks, state, and action predictions, forcing the network to learn characteristics of environment; and also, to detect which of them are valuable for the task. These two contributions were implemented in the DeepMind Lab environment simulating an agent moving inside two different maze scenarios. The results obtained reveal a promising extension of the inclusion of biological-plausible mechanisms inside artificial intelligence applications. Moreover, to include auxiliary tasks improves the performance adding robustness to the system.