Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations

By Michael Harradon, Jeff Druce, Brian E. Ruttenberg. 2018

88 citations recorded by the source corpus.

Deep neural networks are complex and opaque. As they enter application in a variety of important and safety critical domains, users seek methods to explain their output predictions. We develop an approach to explaining deep neural networks by constructing causal models on salient concepts contained in a CNN. We develop methods to extract salient concepts throughout a target network by using autoencoders trained to extract human-understandable representations of network activations. We then build a bayesian causal model using these extracted concepts as variables in order to explain image classification. Finally, we use this causal model to identify and visualize features with significant causal influence on final classification.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, methodology): “iew of interpretable models and their categories Traditional Interpretability Interpretable Models: [106], [103], [64], [102], [65], [105], [50], [13], [40] Causal Interpretability Model-based: [77], [38], [12], [8], [112], [81], [7], Example-based: [35], [45], [39], [101], [76], [86], [73], [61], [”
DARPA's Explainable Artificial Intelligence (XAI) Program (2019, methodology): “For data analytics, CAMEL has been demonstrated using pedestrian detection (using the INRIA pedestrian data set) (Harradon et al. 2018), and CRA is working toward activity recognition tasks (using ActivityNet).”
Generative causal explanations of black-box classifiers (2020, background): “construct explanations based on latent factors, but these explanations are specific to neural network classifiers and require knowledge of the classifier network architecture [38].”
Explaining Deep Learning Models using Causal Inference (2018, methodology): “In (Harradon, Druce, and Ruttenberg 2018), a method is developed to build a causal model over human un-derstandable image based abstractions of the model, which helps in asking counterfactual questions.”
Evaluating Explanations by Cognitive Value (2018, unknown): “Causal models are amenable towards providing explanations as they naturally uncover the cause-eﬀect relationship [37].”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, background): “Recent studies have explored translating deep learning models into decision trees [136], rule-based models [137], or causal models [138].”