Deep Visual MPC-Policy Learning for Navigation

By Noriaki Hirose, Fei Xia, Roberto Mart'in-Mart'in, Amir Sadeghian, Silvio Savarese. 2019

123 citations recorded by the source corpus.

Humans can routinely follow a trajectory defined by a list of images/landmarks. However, traditional robot navigation methods require accurate mapping of the environment, localization, and planning. Moreover, these methods are sensitive to subtle changes in the environment. In this letter, we propose PoliNet, a deep visual model predictive control-policy learning method that can perform visual navigation while avoiding collisions with unseen objects on the navigation path. PoliNet takes in as input a visual trajectory and 360$^{\circ }$ images from robot's current view and outputs velocity commands for a planning horizon of $N$ steps that optimally balance between trajectory following and obstacle avoidance. PoliNet is trained using a differentiable neural image predictive model and a traversability estimation model in an model predictive control setup, with minimal human supervision. PoliNet can be applied to visual trajectory in new scenes without retraining. We show experimentally that the robot can follow a visual trajectory even if it does not start from the exact same position and in the presence of previously unseen obstacles. We validated our algorithm with tests both in a realistic simulation environment and in the real world outperforming state-of-the-art baselines under similar conditions in success rate, coverage rate of the trajectory, and with lower computational load. We also show that we can generate visual trajectory in simulation and execute the corresponding path in the real environment.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

BADGR: An Autonomous Self-Supervised Learning-Based Navigation System (2020, background): “Prior works have investigated learning navigation policies directly from real-world experience, but typically require a person [30, 33, 34] or SLAM algorithm [35] to gather the data, assume access to the ground-truth robot state [36], learn using low-bandwidth sensors [37], or only perform collision”
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action (2022, methodology): “Self-supervised methods for navigation [6–11 , 51 ] instead Large language models are generative models based on the Trans-former architecture [53], trained on large corpora of internet text.”
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, background): “It has been extensively adopted to train visuo-motor policies that successfully transfer to the real world [38, 39, 40, 41], and was the platform for”
ViNT: A Foundation Model for Visual Navigation (2023, methodology): “Following a large body of research in visual navigation, we use a combination of topological graphs for maintaining a spatial representation of the environment and learned policies for low-level control [23– 28], and use learned heuristics to guide the robot in novel environments [15, 29].”
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments (2021, unknown): “Previous works have demonstrated good results developing solutions in iGibson 2.0 that could transfer to real world [2, 85], and evaluated the similarities between simulated and real-world sensor signals [22].”
Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments (2019, background): “They facilitated the development and benchmarking of vision-based (learned) navigation algorithms, and some even enabled to deploy on real robots the algorithms developed in simulation [12], [13].”