Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention

By Jinkyu Kim, John F. Canny. IEEE International Conference on Computer Vision, 2017

377 citations recorded by the source corpus.

Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-tointerpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers etc., can understand what triggered a particular behavior. Here we explore the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network’s output (steering control). Our approach is two-stage. In the first stage, we use a visual attention model to train a convolution network endto- end from images to steering angle. The attention model highlights image regions that potentially influence the network’s output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network’s behavior. We demonstrate the effectiveness of our model on three datasets totaling 16 hours of driving. We first show that training with attention does not degrade the performance of the end-to-end network. Then we show that the network causally cues on a variety of features that are used by humans while driving.

View Paper: https://www.semanticscholar.org/paper/e30e29b3bf4bc19194bad081badeb35803ee2118

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models (2024, background): “HAD [26] employs natural language commands to produce salient maps from drivers’ gaze data.”
DroNet: Learning to Fly by Driving (2018, background): “An advantage with respect to other CNN-based controllers [6], [9], [11]–[13] is, that we can leverage the large body of literature present on steering angle estimation [16], [17] on both the data and the algorithmic point of view.”
DriveGPT4: Interpretable End-to-End Autonomous Driving Via Large Language Model (2023, unknown): “Interpretable Autonomous Driving: To address the black box issue in learning-based autonomous driving, some studies employ visualizations [32], [33], [34].”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “In [33], [208], [211], [218], [219], a learned attention weight is applied to aggregate important features from intermediate feature maps.”
DARPA's Explainable Artificial Intelligence (XAI) Program (2019, methodology): “For autonomy, DEXAI will be demonstrated in vehicle control (using the Berkeley Deep Drive data set and the CARLA simulator) (Kim and Canny 2017) and strategy game scenarios (StarCraft II).”
Unsupervised Real-to-Virtual Domain Unification for End-to-End Highway Driving (2018, background): “Intermediate representations such as semantic segmentation masks and attention maps are shown to be helpful to improving the performance[33, 18].”
Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning (2020, methodology): “Kim et al. [26] used a visual attention model with a causal ﬁlter to visualize the attention heatmap.”
CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving (2018, background): “A line of researches [8,9,10,11,12,13] for learning policies follow the end-to-end imitation learning that directly maps sensor inputs to vehicle control commands via supervised training on large amounts of human driving data.”
PALM: Machine Learning Explanations For Iterative Debugging (2017, background): “Another more recent approach in computer vision is to use attention models, enforce that the model focuses on certain features, for ex-plainability [9].”
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks (2017, methodology): “Another recent work [4], focusing on interpretability for self-driving cars, trained a visual attention model followed by a CNN model to obtain potentially salient image regions and applied causal filtering to find true input regions that actually influence the output.”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “Their approach is two-fold: They add a custom layer that produces attention heatmaps similar to those from (Kim and Canny, 2017); a second custom part uses these heatmaps to generate textual explanations of the decision, which are (weakly) aligned with the model processing. ProtoPNet (Chen et al, 20”
Actively Semi-Supervised Deep Rule-based Classifier Applied to Adverse Driving Scenarios (2019, background): “Consequently, systems need to provide transparency, interpretability, and explainability in order to support specialists and be acceptable by the public [19].”
Multimodal End-to-End Autonomous Driving (2019, background): “terms of its modules; which is more difficult for pure endto-end driving models [78], [79], [80].”
Textual Explanations for Self-Driving Vehicles (2018, methodology): “An alternative to regularization, [11] use causal filtering over the controller’s attention maps and achieve about 60% reduction in “hot” attention pixels.”
Explanations in Autonomous Driving: A Survey (2021, unknown): “[95] 3 3 3 3 3 UG A & C P Kim & Canny [96] 3 3 3 3 3 3 3 UG B & A P Cultrera et al.”
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving (2023, background): “In the field of autonomous vehicles, visual attention maps which highlight causally influential regions in driving images were proposed in [24].”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “As feature maps preserve spatiality, visual attention can be applied on the feature maps from the convolutional layers [40], [89].”
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars (2018, methodology): “In our evaluation we do not consider [8], since, in spite of offering an interpretable solution, it gives almost no improvements over the simpler architecture in [15].”