Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots

By Lei Tai. 2016

114 citations recorded by the source corpus.

Exploration in an unknown environment is the core functionality for mobile robots. Learning-based exploration methods, including convolutional neural networks, provide excellent strategies without human-designed logic for the feature extraction. But the conventional supervised learning algorithms cost lots of efforts on the labeling work of datasets inevitably. Scenes not included in the training set are mostly unrecognized either. We propose a deep reinforcement learning method for the exploration of mobile robots in an indoor environment with the depth information from an RGB-D sensor only. Based on the Deep Q-Network framework, the raw depth image is taken as the only input to estimate the Q values corresponding to all moving commands. The training of the network weights is end-to-end. In arbitrarily constructed simulation environments, we show that the robot can be quickly adapted to unfamiliar scenes without any man-made labeling. Besides, through analysis of receptive fields of feature representations, deep reinforcement learning motivates the convolutional networks to estimate the traversability of the scenes. The test results are compared with the exploration strategies separately based on deep learning or reinforcement learning. Even trained only in the simulated environment, experimental results in real-world environment demonstrate that the cognitive ability of robot controller is dramatically improved compared with the supervised method. We believe it is the first time that raw sensor information is used to build cognitive exploration strategy for mobile robots through end-to-end deep reinforcement learning.

Subfields: Artificial Intelligence, Computer Networks and Communications, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (2017, background): “RL in navigation A number of recent papers use reinforcement learning (RL) to train navigational agents [25, 41, 44, 51], although these works do not address language instruction, which is the focus of this paper.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “AN BE APPLIED TO CONTINOUS CONTROL AND THEIR TRAINING EFFICIENCY Approach Value Policy Model Continuous Training Application Estimation Optimization Requirement Control Efﬁciency DQN X F Exploration (Tai and Liu, 2016c) (Mnih et al., 2015) Motion Control (Zhang et al., 2015) DDPG X X X F F UAV contr”
Learning to Navigate in Complex Environments (2016, background): “Tai & Liu (2016) trained a convnet DQN-based agent using depth channel inputs for obstacle avoidance in 3D environments.”
Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, neutral): “This observation is similar to other works which use a diverse set of input modalities (Mirowski et al., 2016; Tai & Liu, 2016).”
Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation (2017, background): “In terms of mobile robots, the complicated environments enlarge the sample space extremely while deep-RL methods normally sample the action from a discrete space to simplify the problem [4], [5], [6].”
Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not (2016, methodology): “For example in our previous work (Tai and Liu, 2016b), a deep-learning model was used for the homing vector prediction in the mobile robot visual homing problem.”