Watch this: Scalable cost-function learning for path planning in urban environments

By Markus Wulfmeier, Dominic Zeng Wang, Ingmar Posner. 2016

120 citations recorded by the source corpus.

In this work, we present an approach to learn cost maps for driving in complex urban environments from a large number of demonstrations of human driving behaviour. The learned cost maps are constructed directly from raw sensor measurements, bypassing the effort of manually designing cost maps as well as features. When deploying the cost maps, the trajectories generated not only replicate human-like driving behaviour but are also demonstrably robust against systematic errors in putative robot configuration. To achieve this we deploy a Maximum Entropy based, non-linear IRL framework which uses Fully Convolutional Neural Networks (FCNs) to represent the cost model underlying expert driving behaviour. Using a deep, parametric approach enables us to scale efficiently to large datasets and complex behaviours while being run-time independent of dataset extent during deployment. We demonstrate scalability and performance on an ambitious dataset collected over the course of one year including more than 25k demonstration trajectories extracted from over 120km of driving and 13 different drivers. We evaluate against a carefully designed cost map and, in addition, demonstrate robustness to systematic errors by learning precise cost-maps even in the presence of system calibration perturbations.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “015) used a deep architecture to train the reward function based on Maximum Entropy paradigm for IRL. And it was successfully applied to a cost-map learning work for the mobile vehicle path planning (Wulfmeier et al., 2016). Finn et al. (2016) taught a high-dimensional robot system to learn behavior”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, background): “Much of the prior work on learning-based robot navigation has focused on imitation learning [19, 20, 21, 22, 23], which requires demonstrations.”
Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations (2018, methodology): “[12] present a similar approach using deep IRL instead of a combination of classical features in order to learn how to drive an autonomous car through static environments.”
Human Action Recognition and Prediction: A Survey (2018, methodology): “This method also extends inverse optimal control (IOC) [181], [184] to a deep model, which has shown promising results in robot control [197] and driving [198] tasks.”
Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not (2016, methodology): “And it was successfully applied to a cost-map learning work for the mobile vehicle path planning (Wulfmeier et al., 2016).”
Few-Shot Goal Inference for Visuomotor Learning and Planning (2018, background): “Other works have sought to learn objectives by training a classiﬁer [12, 13, 14] or reward function [10, 24, 11], including in the framework of inverse RL [25, 7, 8, 26, 27, 9, 28, 29, 30].”
Learning a Prior over Intent via Meta-Inverse Reinforcement Learning (2018, methodology): “In order to learn from high dimensional input, Wulfmeier et al. (2015) proposed applying fully convolutional networks (Shelhamer et al., 2017) to the MaxEnt IRL framework (Ziebart et al., 2008) for several navigation tasks (Wulfmeier et al., 2016a;b).”
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents (2017, background): “Notice that unlike typical robotics applications [50, 11], we do not assume that the goal (final destination) is known or the dynamics of the agents are given.”
A Survey of State-Action Representations for Autonomous Driving (2018, unknown): “This representation is used by (Ziebart et al., 2009; Richter et al., 2014; Chen et al., 2016; Tamar et al., 2016; Shankar et al., 2016; Wulfmeier et al., 2016; Sallab et al., 2017; Rehder et al., 2017a; Hoermann et al., 2017; Williams et al., 2017; Mukadam et al., 2018; Rhinehart et al., 2018).”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, methodology): “Zhu et al. [189] use RL ConvNet and state visiting frequency (SVF) ConvNet to encode the vehicle’s kinematics and obtain the weight of the reward function by back-propagating the loss gradient [193] between expert SVF from expert demonstration and policy SVF from lidar data.”
A survey of deep learning techniques for autonomous driving (2019, background): “Learning (IRL) [27], to learn from human driving demonstrations without needing to explore unsafe actions.”
Feature Expansive Reward Learning: Rethinking Human Input (2020, methodology): “A natural answer is in deep IRL methods (Wulfmeier et al., 2016; Finn et al., 2016; Levine et al., 2011), which learn rewards defined directly on the high-dimensional raw state (or observation) space, thereby constructing features automatically.”