Driving Policy Transfer via Modularity and Abstraction

By Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun. 2018

245 citations recorded by the source corpus.

End-to-end approaches to autonomous driving have high sample complexity and are difficult to scale to realistic urban driving. Simulation can help end-to-end driving systems by providing a cheap, safe, and diverse training environment. Yet training driving policies in simulation brings up the problem of transferring such policies to the real world. We present an approach to transferring driving policies from simulation to reality via modularity and abstraction. Our approach is inspired by classic driving systems and aims to combine the benefits of modular architectures and end-to-end deep learning approaches. The key idea is to encapsulate the driving policy such that it is not directly exposed to raw perceptual input or low-level vehicle dynamics. We evaluate the presented approach in simulated urban environments and in the real world. In particular, we transfer a driving policy trained in simulation to a 1/5-scale robotic truck that is deployed in a variety of conditions, with no finetuning, on two continents. The supplementary video can be viewed at this https URL

Subfields: Automotive Engineering, Building and Construction, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Building and Construction

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

A General Framework for Uncertainty Estimation in Deep Learning (2019, methodology): “End-to-End Steering Angle Prediction: Neural networks trained to predict steering angles from images have been used in several robotics applications, e.g. autonomous driving [29], [30] and ﬂying [31].”
On Evaluation of Embodied Navigation Agents (2018, background): “For this reason, we advocate broad adoption of simulation platforms by the community, accompanied by continued investment in techniques for transferring learned navigation skills from simulation to reality [26, 21].”
MP3: A Unified Model to Map, Perceive, Predict and Plan (2021, background): “Subsequent to the success of deep learning, direct control based methods have advanced with deeper networks, richer sensors, and scalable learning methods [5,11,24,35].”
Learning by Cheating (2019, background): “They can be transferred to the physical world using any procedure for sim-to-real transfer [4, 14].”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, methodology): “Muller et al. [32] focus on lane keeping and making turns in outdoor environments using a scene segmentation module which is used to generate the next desired position for the vehicle based on a user command.”
Neural Autonomous Navigation with Riemannian Motion Policy (2019, background): “[22] presents an approach that decouples perception and control by predicting a waypoint from a segmented image, whereby a PID controller is used to control the vehicle.”
Challenges in Designing Datasets and Validation for Autonomous Driving (2019, background): “, 2017), (Mueller et al., 2018), can be useful as potential mitigation of the lack of data.”
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach (2021, background): “This is feasible because on the one hand, BEV as a strong abstraction reduces the sim-to-real gap [31], and on the other hand, on-policy labeling does not have to happen in real-time or even onboard.”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, methodology): “The described method [78] was tested on a real vehicle and outperformed other domain adaptation methods, including simulation-to-real [46] and realto-simulation [77] transformation.”
Learning high-speed flight in the wild (2021, methodology): “We leverage abstraction of the input data to transfer the policy from simulation to reality (28, 29).”
RoboTHOR: An Open Simulation-to-Real Embodied AI Platform (2020, unknown): “The main approaches are based on randomization of the source domain to better generalize to the target domain [63, 29, 51, 44, 61], learning the mapping between some abstraction or higher order statistics of the source and target domains [27, 59, 19, 35, 76, 42, 78], interpolating between the source”
Conditional Affordance Learning for Driving in Urban Environments (2018, background): “Finally, inspired by recent work [42], we plan to transfer our results from simulation to the real world.”
End-To-End Interpretable Neural Motion Planner (2019, unknown): “[21] incorporate road segmentation as intermediate representations, which are then converted into steering commands.”
Learning to Drive from Simulation without Real World Labels (2018, methodology): “[19] transferred policies from simulation to reality by using a modular architecture predicting control decisions learned from semantic image segmentation.”
Deep Local Trajectory Replanning and Control for Robot Navigation (2019, methodology): “[42], we avoid providing complex sensory input, like images, to our method.”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, background): “Robotics is a particularly inspiring domain for sim2real research, and here it has been again found that more abstract representations than RGB, such as segmentations [50, 94], can improve performance.”
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation (2020, background): “Researchers have proposed learning-based solutions that map partial observations to either waypoints [22, 23, 24, 25] or whole trajectories [26], thus bypassing the trajectory searching problem.”
Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning (2020, background): “…in these fast and efficient environments, whether it is to solve complex computer games (Dosovitskiy & Koltun, 2017; Jaderberg et al., 2019; Vinyals et al., 2019) or sophisticated robotic control problems via sim-to-real transfer (Müller et al., 2018; 1Intel Labs 2University of Southern California.”
SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation (2019, methodology): “[22] we use auxiliary intermediate objectives to aid in transfer, but in our case focus on a set of auxiliary visual and motion tasks which generalize to many downstream embodied tasks and propose techniques to selectively transfer either across visual environments or across embodied tasks.”
Exploring the Limitations of Behavior Cloning for Autonomous Driving (2019, methodology): “Note, however, that transferring policies from simulation to the real-world is an open problem [27] out of the scope of this paper, although recent works have shown encouraging results [29, 44].”