Interpretable Machine Learning

By Bradley C. Boehmke, Brandon M. Greenwell. 2019

3,181 citations recorded by the source corpus.

Interpretable machine learning has become a popular research direction as deep neural networks (DNNs) have become more powerful and their applications more mainstream, yet DNNs remain difficult to understand. Testing with Concept Activation Vectors, TCAV, (Kim et al. 2017) is an approach to interpreting DNNs in a human-friendly way and has recently received significant attention in the machine learning community. The TCAV algorithm achieves a degree of global interpretability for DNNs through human-defined concepts as explanations. This project introduces Robust TCAV, which builds on TCAV and experimentally determines best practices for this method. The objectives for Robust TCAV are 1) Making TCAV more consistent by reducing variance in the TCAV score distribution and 2) Increasing CAV and TCAV score resistance to perturbations. A difference of means method for CAV generation was determined to be the best practice to achieve both objectives. Many areas of the TCAV process are explored including CAV visualization in low dimensions, negative class selection, and activation perturbation in the direction of a CAV. Finally, a thresholding technique is considered to remove noise in TCAV scores. This project is a step in the direction of making TCAV, an already impactful algorithm in interpretability, more reliable and useful for practitioners.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Association Between Labels and Free-Text Rationales (2020, background): “Interpretable NLP aims to better understand predictive models’ internals for purposes such as de-bugging, validating safety before deployment, or revealing unintended biases and behavior (Molnar, 2019).”
WT5?! Training Text-to-Text Models to Explain their Predictions (2020, methodology): “Instead, we refer the interested reader to the surveys provided by (Doshi-Velez & Kim, 2017; Molnar, 2019; Guidotti et al., 2018).”
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, neutral): “…Hoffman, Mueller, and Klein, 2017, 2018; Kass and Finin, 1988; Lacave and Díez, 2002; Lipton, 2016; Miller, 2017; Miller, Howe, and Sonenberg, 2017; Molnar, 2018; Moore and Swartout, 1988; Moulin, Irandoust, Bélanger, and Desbordes, 2002; Nakatsu, 2004; Sørmo, Cassens, and Aamodt, 2005; Swartout,…”
On the Art and Science of Machine Learning Explanations (2018, background): “• Because tree shap explanations are offsets from a global intercept, each φ j can be interpreted as the difference in g ( x ) and the average of g ( X ) associated with some input feature x j [21].”
Explainable Recommendation: A Survey and New Perspectives (2018, background): “Explainable recommendation models can either be model-intrinsic or model-agnostic (Lipton, 2018; Molnar, 2019).”
Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward (2020, background): “Example-based explanations (Molnar, 2020) may also contribute to an effective engagement of all the parties by helping in bridging technical divides across developers, experts in other ﬁ elds, and lay-people.”
Do Not Trust Additive Explanations (2019, background): “, 2018; Alvarez-Melis and Jaakkola, 2017), and tabular data (Molnar, 2019; Biecek, 2018).”
Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, background): “ﬀerent for samples with the same label with a slightly diﬀerent features. This instability could be the result of a high variance as well as the non-deterministic components of the explanation method [72]. 5.2.2 Causal Evaluation Metrics Due to the lack of groundtruth for causal explanations, to ver”
Exploring Example Influence in Continual Learning (2022, positive): “In recent years, as the impressive Interpretable Machine Learning (IML) [27] develops, people realize the importance of exploring the nature of data-driven machine learning.”
Trust in AI: progress, challenges, and future directions (2024, neutral): “Despite these challenges, it is hoped that interpretability will soon reach a state of readiness(Molnar et al., 2020).”
RAPID: Rapid and Precise Interpretable Decision Sets (2019, background): “Further, as argued in [5] and [43], sets of decision rules are the most interpretable of all machine learning models.”
Black Box Explanation by Learning Image Exemplars in the Latent Feature Space (2019, background): “increasing interest in the scientific community in deriving explanations able to describe the behavior of a black box [5,22,13,6], or explainable by design approaches [19,18].”
Explainable Deep Learning: A Field Guide for the Uninitiated (2020, neutral): “Strong examples are given by Molnar (2020).”
On Explaining Decision Trees (2020, background): “DTs embody what is widely regarded as an interpretable ML model [20,52,42,60,41,39,23,77,62].”