Learning by Cheating

By Dian Chen, Brady Zhou, Vladlen Koltun, Philipp Krahenbuhl. 2019

689 citations recorded by the source corpus.

Vision-based urban driving is hard. The autonomous system needs to learn to perceive the world and act in it. We show that this challenging learning problem can be simplified by decomposing it into two stages. We first train an agent that has access to privileged information. This privileged agent cheats by observing the ground-truth layout of the environment and the positions of all traffic participants. In the second stage, the privileged agent acts as a teacher that trains a purely vision-based sensorimotor agent. The resulting sensorimotor agent does not have access to any privileged information and does not cheat. This two-stage training procedure is counter-intuitive at first, but has a number of important advantages that we analyze and empirically demonstrate. We use the presented approach to train a vision-based autonomous driving system that substantially outperforms the state of the art on the CARLA benchmark and the recent NoCrash benchmark. Our approach achieves, for the first time, 100% success rate on all tasks in the original CARLA benchmark, sets a new record on the NoCrash benchmark, and reduces the frequency of infractions by an order of magnitude compared to the prior state of the art. For the video that summarizes this work, see this https URL

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning quadrupedal locomotion over challenging terrain (2020, positive): “One difference of our methodology from that of Chen et al. [23] is that we do not rely on expert demonstrations to train the privileged policy; rather, the teacher policy is trained via reinforcement learning.”
Parting with Misconceptions about Learning-based Vehicle Motion Planning (2023, methodology): “Ego-forecasting approaches include both end-to-end methods that utilize LiDAR scans [40, 41], RGB images [42, 43, 44, 45, 14, 46] or both [13, 47, 5], as well as modular methods involving lower-dimensional inputs like bird’s eye view (BEV) grids or state vectors [26, 48, 11, 49, 50].”
Planning-oriented Autonomous Driving (2022, background): “Pioneering works [7, 9, 14, 15, 45, 53, 54, 60] verified this vanilla design in the closed-loop simulation [17].”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, background): “3) Inverse Reinforcement Learning: Inverse Reinforcement Learning (IRL) [33], [74] aims to deduce the underlying spe-ciﬁc behaviours through the reward function.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “LBC[5]adoptsDAggerandresamples thedatawithhigherlossmorefrequently.”
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline (2022, background): “LBC [11] and NEAT [15] decode waypoints from a dense heatmap or offset map.”
Learning high-speed flight in the wild (2021, background): “We train the navigation policy via privileged learning [30] on demonstrations that are provided by a sampling-based expert.”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, methodology): “This 264 technique has been fruitfully applied to self-driving [80, 81], autonomous drone flight [82, 83], 265 among others.”
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer (2022, methodology): “LBC [16] proposed the mimicking methods aimed at training image-input networks with the supervision of a privileged model or squeezed model.”
Imitation Learning: A Survey of Learning Methods, Environments and Metrics (2024, unknown): “One common approach in these work [8, 56, 57, 64] is coupling behavioural cloning with other learning mechanisms, such as attention [57] or domain-shift [8, 56].”
A System for General In-Hand Object Re-Orientation (2021, background): “To demonstrate the possibility of transferring learned policies to the real world in the future, we overcome the need for privileged information using the idea of teacher-student training [14, 15].”
LMDrive: Closed-Loop End-to-End Driving with Large Language Models (2023, methodology): “To get the final control signal, which includes braking, throttling, and steering, following LBC [4], we use two PID controllers for latitudinal and longitudinal control to track the heading and velocity of predicted waypoints respectively.”
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach (2021, methodology): “LBC [9] maps camera images to waypoints by mimicking a privileged mid-to-mid IL agent similar to Chauffeurnet [3], which takes BEV as input and outputs future waypoints.”
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking (2024, unknown): “For instance, an extensive body of end-to-end approaches focuses on closed-loop simulators, utilizing single-frame cameras, LiDAR point clouds, or a combination of both for expert imitation [7, 11, 36, 8, 52, 43, 44, 12, 24, 57, 22].”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, methodology): “We consider the task of point-to-point navigation in an urban setting [1], [2], [38], [86], [87] where the goal is to complete a given route while safely reacting to other dynamic agents and following trafﬁc rules.”
ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning (2022, background): “One direction of end-to-end pipelines is to focus on the ultimate planning task mainly without explicit design of the intermediate representation [13, 14, 42, 9, 12, 26].”