Conditional Affordance Learning for Driving in Urban Environments

By Axel Sauer, Nikolay Savinov, Andreas Geiger. 2018

197 citations recorded by the source corpus.

Most existing approaches to autonomous driving fall into one of two categories: modular pipelines, that build an extensive model of the environment, and imitation learning approaches, that map images directly to control outputs. A recently proposed third paradigm, direct perception, aims to combine the advantages of both by using a neural network to learn appropriate low-dimensional intermediate representations. However, existing direct perception approaches are restricted to simple highway situations, lacking the ability to navigate intersections, stop at traffic lights or respect speed limits. In this work, we propose a direct perception approach which maps video input to intermediate representations suitable for autonomous navigation in complex urban environments given high-level directional inputs. Compared to state-of-the-art reinforcement and conditional imitation learning approaches, we achieve an improvement of up to 68 % in goal-directed navigation on the challenging CARLA simulation benchmark. In addition, our approach is the first to handle traffic lights and speed signs by using image-level labels only, as well as smooth car-following, resulting in a significant reduction of traffic accidents in simulation.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking (2024, unknown): “Second, we draw inspiration from the literature of rule-based planning for AVs [41, 18, 39, 16] to identify a set of diverse, efficient, and principled metrics that cover multiple facets of the autonomous driving task.”
Deep Object-Centric Policies for Autonomous Driving (2018, unknown): “Unlike in [11], [13], we do not use any data augmentation.”
Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning (2020, methodology): “Sauer et al. [27] analyzed the decision making process of the deep neural network by using gradient-weighted class activation maps to obtain the attention of the network.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “In [192], [193], predicted affordance indicators, such as trafﬁc light states, offset to the lane center, and distance to the leading vehicle, are used as representations for policy learning.”
Parting with Misconceptions about Learning-based Vehicle Motion Planning (2023, methodology): “Ego-forecasting approaches include both end-to-end methods that utilize LiDAR scans [40, 41], RGB images [42, 43, 44, 45, 14, 46] or both [13, 47, 5], as well as modular methods involving lower-dimensional inputs like bird’s eye view (BEV) grids or state vectors [26, 48, 11, 49, 50].”
End-To-End Interpretable Neural Motion Planner (2019, background): “[26] further propose a variant conditioned on direction command.”
Learning by Cheating (2019, background): “Task Weather MP[8] CIL[6] CIRL[13] CAL[20] CILRS[7] LBC LBC† Straight train 92 97 100 93 96 100 100 One turn 61 59 71 82 84 100 100 Navigation 24 40 53 70 69 100 98 Nav.”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, methodology): “To model temporal aspects, the model needs to consider the combination of multiple past frames [8], [47], [48], [56], [84].”
Conditional Vehicle Trajectories Prediction in CARLA Urban Environment (2019, background): “Conditional Affordance Learning (CAL) [30] adapts this to direct perception, and improves over CIL.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “While the modular approach is the common pipeline for autonomous driving, some works also try to map the sensor data directly to the decision policy such as steering angles or pedal positions (end-to-end learning) [243]–[245], or to some intermediate environment representations (direct-perception) [”
Multimodal End-to-End Autonomous Driving (2019, methodology): “In order to answer the posed question, as is being common practice in many recent previous works on end-to-end driving [71], [68], [67], [69], [72], [73], [80], we will rely on the open source driving simulator known as CARLA [98] and its standard benchmark.”
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst (2018, methodology): “The CARLA simulator (Dosovitskiy et al. (2017)) has enabled recent work such as Sauer et al. (2018), which estimates several aﬀordances from sensor inputs to drive a car in a simulated urban environment.”