Extracting Tree-Structured Representations of Trained Networks

By Mark W. Craven, Jude W. Shavlik. 1995

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

On the Art and Science of Machine Learning Explanations (2018, methodology): “This type of model is referred to by various other names, such as proxy or shadow models and the process of training surrogate models is sometimes referred to as model extraction [8], [30], [5].”
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge (2018, methodology): “These include compressing neural networks into simple models, e.g. a small decision tree, which is easy to interpret (Craven & Shavlik, 1996), retrieving typical inputs so that one can interpret by examples via hidden activations (Caruana et al., 1999) or inﬂuence functions (Koh & Liang, 2017) and…”
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability (2017, background): “Craven and Shavlik (1996) train decision trees that mimic the predictions of a ﬁxed, pretrained neural network, but do not train the network itself to be simpler.”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, methodology): “We assess fidelity, the notion that the student model should match the teacher model [10], using the closeness of the student model to the teacher’s predictions.”
MLIC: A MaxSAT-Based Framework for Learning Interpretable Classification Rules (2018, background): “A useful generalization is a “K-of-N” clause [13], which is true when at least K of the N literals are satisfied.”
Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees (2018, methodology): “Rule extraction is a common method to extract knowledge from tree models [7,6,2].”
Explaining Explanations in AI (2018, background): “Historically, much work has focused on global approximations of models (e.g. Craven and Shavlik (1996); Martens et al. (2007); Sanchez et al. (2015), including approaches based on clustering (Chen et al., 2016), integer programming (Zeng et al., 2017), and rule lists (Wang and Rudin, 2015)).”
Programs as Black-Box Explanations (2016, background): “One of the most commonly used interpretable representations is that of decision trees [3].”
Meaningful Explanations of Black Box AI Decision Systems (2019, methodology): “…training set is available, decision rules are also widely used to proxy a black box model by directly designing a transparent classiﬁer (Guidotti et al. 2018b) which is locally or globally interpretable on its own (Lakkaraju, Bach, and Leskovec 2016; Malioutov et al. 2017; Craven and Shavlik 1995).”
Rule extraction from support vector machines: A review (2010, methodology): “Table 1 summarizes the similarities between ANNs and SVMs [52,58,64].”
Interpreting Shared Deep Learning Models via Explicable Boundary Trees (2017, methodology): “Another common approach is to mimic a complicated model using an interpretable simple model such as decision tree (Craven and Shavlik 1996; Schmitz, Aldrich, and Gouws 1999; Boz 2002; Che et al. 2016).”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, methodology): “Many IML methods are surrogate model approaches [89,75,72,95,34,10,18,61] and differ, e.”
Explainability fact sheets: a framework for systematic assessment of explainable approaches (2019, methodology): “We are currently working on creating these Fact Sheets for popular explainability techniques such as surrogates [5, 43] and coun-terfactuals [34].”
Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes (2012, background): “For discussion, see Craven and Shavlik (1996). A-learning represents a middle ground between Q learning and these approaches in that it allows for flexible modeling of the functions hk(s̄k, āk−1) while maintaining simple parametric models for the contrast functions Ck(s̄k, āk−1).”