Transparent Model Distillation

By Sarah Tan, R. Caruana, Giles Hooker, Albert Gordo. 2018

38 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

FEED: Feature-level Ensemble for Knowledge Distillation (2019, background): “This motivated researchers to develop many variants of it to various domains, and many researchers studied ways to better teach the student (Frosst and Hinton 2017; Pham et al. 2018; Radosavovic et al. 2018; Tan et al. 2018; Tarvainen and Valpola 2017).”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “Interested researchers can also consult these [125]–[127] for further details about this technique.”
Interpreting CNNs via Decision Trees (2018, background): “Decision trees for neural networks: Distilling knowledge from neural networks into tree structures is an emerging direction [13, 31, 5], but the trees did not explain the network knowledge at a human-interpretable semantic level.”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, background): “We point the reader to [9, 24, 25] to learn more about iGAMs and to [31] for a distillation example where it was used as a student.”
High-Accuracy Low-Precision Training (2018, background): “It has been folklore for many years that neural network inference could be done effectively even with 8-bit arithmetic [31], and there has been much work recently on compressing already-trained networks by (among other things) making some of the weights and activations low-precision [3, 12, 30].”
Improving the Interpretability of Deep Neural Networks with Knowledge Distillation (2018, background): “Another approach that distills neural networks into GBTs is in [42].”
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations (2019, methodology): “…inherently interpretable models (Letham et al., 2015; Lakkaraju et al., 2016), and creating post-hoc methods to explain black-box models, either locally around a speciﬁc input (Baehrens et al., 2010; Ribeiro et al., 2016) or globally for the entire model (Ribeiro et al., 2018a; Tan et al., 2018a).”
Interpretable Approaches to Detect Bias in Black-Box Models (2018, unknown): “A preprint can be found at [17].”
Please Stop Permuting Features: An Explanation and Alternatives (2019, methodology): “Tan et al. (2018) found that these better represented the over-all behavior of the function than methods based on combining local explanations.”
Explainable Neural Networks based on Additive Index Models (2018, background): “These models which may be either global (Hinton et al. (2015), Bucilua et al. (2006), Tan et al. (2018)) or local (Hu et al. (2018)) in nature.”
MAGIX: Model Agnostic Globally Interpretable Explanations (2017, background): “, 2017) and (Tan et al., 2018) investigate how model distillation can be used to distill complex models into into models that are transparent or interpretable in some sense.”
Interpretability is Harder in the Multiclass Setting: Axiomatic Interpretability for Multiclass Additive Models (2018, background): “The first category tries to explain predictions of a black-box model, either locally (Ribeiro, Singh, and Guestrin 2016; Baehrens et al. 2010) or globally (Ribeiro, Singh, and Guestrin 2018; Tan et al. 2018).”
Born Again Neural Networks (2018, background): “, 2007) and generalized additive models (Tan et al., 2018) for the purpose of increasing transparency or interpretability.”
Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP) (2018, methodology): “In the ML literature, it is referred to as model distillation (Tan, Caruana, Hooker, & Gordo, 2018) and model compression (Bucilua, Caruana, & Niculescu-Mizil, 2006).”