Textual Explanations for Self-Driving Vehicles

By Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John F. Canny, Zeynep Akata. 2018

460 citations recorded by the source corpus.

Deep neural perception and control networks have become key components of self-driving vehicles. User acceptance is likely to benefit from easy-to-interpret textual explanations which allow end-users to understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller’s output, namely rationalizations. We propose a new approach to introspective explanations which consists of two parts. First, we use a visual (spatial) attention model to train a convolutional network end-to-end from images to the vehicle control commands, i.e., acceleration and change of course. The controller’s attention identifies image regions that potentially influence the network’s output. Second, we use an attention-based video-to-text model to produce textual explanations of model actions. The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. Finally, we explore a version of our model that generates rationalizations, and compare with introspective explanations on the same video segments. We evaluate these models on a novel driving dataset with ground-truth human explanations, the Berkeley DeepDrive eXplanation (BDD-X) dataset. Code is available at https://github.com/JinkyuKimUCB/explainable-deep-driving.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, methodology): “…paradigm for producing free-text explanations is via direct supervision, i.e., training an autoregressive, generative language model to predict human-authored explanations directly (Kim et al., 2018; Park et al., 2018; Ehsan et al., 2018; Narang et al., 2020; Wiegreffe et al., 2021, i.a. ).”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “…time and generate such explanations for their decisions at deployment time (Hendricks et al., 2016; Ling et al., 2017; Park et al., 2018; Camburu et al., 2018; Kim et al., 2018; Rajani et al., 2019; Camburu et al., 2020; Narang et al., 2020; Kumar and Taluk-dar, 2020; Marasovi´c et al., 2022).”
Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “On the other hand, a standard approach to generating free-text explanations is to use human-written explanations (Hendricks et al., 2016; Kim et al., 2018; Hancock et al., 2018; Camburu et al., 2018; Ehsan et al., 2018; Liu et al., 2019; Wu and Mooney, 2019; Narang et al., 2020; Tang et al., 2020; M”
Measuring Association Between Labels and Free-Text Rationales (2020, background): “This is the most common approach to free-text rationalization (Hendricks et al., 2016; Kim et al., 2018; Camburu et al., 2018; Ehsan et al., 2018; Liu et al., 2019; Wu and Mooney, 2019; Narang et al., 2020; Do et al., 2020), but little is understood about its faithfulness.”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, background): “This has motivated work in having models provide explanations for their behavior, in the form of a natural language sentence [31, 9, 41] or an attention map [32, 35, 37].”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, methodology): “Existing works have proposed to use a learned gaze prediction model to guide the process of learning a human policy [Li et al. , 2018; Zhang et al. , 2018; Kim et al. , 2018].”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, background): “Besides the image classification datasets ACT-X [37] and CUB [43; 23], and the video dataset BDD-X [26], there are currently three VL datasets (i.e., the task itself includes images and text) with NLEs.”
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations (2019, background): “For example, for explanations in self-driving cars [19], one can interchange “green light” with “red light”, or “stop” with “accelerate”, to get inconsistent — and potentially hazardous!”
XOC: Explainable Observer-Classifier for Explainable Binary Decisions (2019, background): “, “The car is accelerating because there are no other car in its lane” [16].”
NILE : Natural Language Inference with Faithful Natural Language Explanations (2020, background): “There has been growing interest in producing natural language explanations for deep learning systems (Huk Park et al., 2018; Kim et al., 2018; Ling et al., 2017), including NLI (Camburu et al., 2018).”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “ai 2019 [140] 1 X X X X 30h MIT DeepDrive [141] 1 X X X X X X 1100h Berkley DeepDrive-X [142] 1 X X X X X 77h Berkley Oxford RobotCar [143] 4 X X X X X 214h CC BY-NC-SA 4.”
Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? (2020, methodology): “Two works focus on multi-modal problems, explaining visual question answering (Park et al., 2018) and self-driving car decisions (Kim et al., 2018).”
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs (2020, background): “…(Wu and Mooney, 2019; Rajani et al., 2019), arcade games (Ehsan et al., 2019), fact checking (Atanasova et al., 2020), image classification (Hendricks et al., 2018), motivation prediction (Vondrick et al., 2016), algebraic word problems (Ling et al., 2017), and self-driving cars (Kim et al., 2018).”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “Kim et al. [33] and Xu et al. [228] develop datasets pairing driving videos or images with descriptions and explanations, and propose end-to-end models with both control and explanation outputs.”
Rationale-Inspired Natural Language Explanations with Commonsense (2021, background): “A growing number of works in natural language processing and computer-vision are focusing on designing neural models that produce NLEs for their predictions to make these models accessible to their users (Hendricks et al., 2016; Camburu et al., 2018; Park et al., 2018; Kim et al., 2018; Ling et al.,”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “[125]∗ sentiment classification crowd + authors 85 E-δ-NLI [18] defeasible natural language inference auto 92,298 (∼8) BDD-X†† [62] vehicle control for self-driving cars crowd ∼26K VQA-E†† [75] visual QA auto ∼270K VQA-X†† [94] visual QA crowd 28,180 (1 or 3) ACT-X†† [94] activity recognition crowd ”
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving (2023, background): “In [25] the authors combined attention based methods with natural language into an attention-based vehicle controller to provide natural language action descriptions and explanations based on a series of image frames.”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “This is especially of interest in applications with a limited time frame for user reaction, like product recommendation systems [101] or automated driving applications [77].”