Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

By Mike Wu, Michael C. Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, Finale Doshi-Velez. 2017

300 citations recorded by the source corpus.

The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep time-series models so their class-probability predictions have high accuracy while being closely modeled by decision trees with few nodes. Using intuitive toy examples as well as medical tasks for treating sepsis and HIV, we demonstrate that this new tree regularization yields models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Interpreting CNNs via Decision Trees (2018, background): “[35] learned a decision tree via knowledge distillation to represent the output feature space of an RNN, in order to regularize the RNN for better representations.”
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, background): “Architectural modifications include regularization to set shape constraints [54], to stimulate high layers”
Network Transplanting (2018, background): “Some recent stud-+ ies [7, 25] distilled network knowledge into decision trees.”
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use (2019, methodology): “Although simpler model classes (Lundberg et al., 2018; Wu et al., 2018) are considered more interpetable by users, a general trade-off between model performance and explainability is assumed (Caruana et al., 2015).”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “Examples of model interpretation techniques include inherently interpretable models [205], deep model simplification [213], and visualization of model internals [215].”
On Explaining Decision Trees (2020, background): “Moreover, there has been work on distilling or approximating complex ML models with (soft) decision trees [21,9,8,74,76,56,75].”
Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation (2019, background): “As demonstrated by previous works [31, 32, 38], prompting the neurons connection being sparse is beneficial for obtaining a well generalization ability.”
Human Evaluation of Models Built for Interpretability (2019, methodology): “Lakkaraju, Bach, and Leskovec, 2016; Wu et al., 2018 who assume the users of interpretable systems are domain experts). Doing such an extensive training was not realistic in this setting, so we used the following strategy: First, participants were given a tutorial on each task and the interface. Nex”
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations (2020, background): “Techniques range from explaining models with linear equations of salient features [40, 47, 53], rules and decision trees [48, 69, 71], partial dependence [31], etc.”