Extracting Rules from Artifical Neural Networks with Distributed Representations

By Sebastian Thrun. 1994

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “Validity Interval Analysis is a type of sensitivity analysis to mimic neural network behavior [31].”
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, neutral): “…Dosovitskiy, Brox, and Riedmiller, 2014; Sturm, Lapuschkinb, Samek, and Müller, 2016; Sutskever, and Hinton, 2012; Tapaswi et al., 2016; Thrun, 1995; Usunier, Synnaeve, Lin, and Chintala, 2016; Xu and Saenko, 2016 Yosinski, et al., 2015; Zeiler and Fergus, 2014; Zeiler, Taylor and…”
Interpretable Neural Predictions with Differentiable Binary Variables (2019, methodology): “For example, Thrun (1995) extract if-then rules, while Craven and Shavlik (1996)”
RISE: Randomized Input Sampling for Explanation of Black-box Models (2018, background): “Another line of research focused on approximating the less interpretable models ( e.g ., neural network, non-linear SVMs etc .) with simple, interpretable models such as decision rules or sparse linear models [3, 28].”
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning (2018, background): “Validity Interval Analysis is a type of sensitivity analysis to mimic neural network behavior [31].”
Extracting comprehensible models from trained neural networks (1996, methodology): “For example, two of rule-extraction methods for real-valued problem domains discussed in the previous chapter (Saito & Nakano, 1988; Thrun, 1995) try to cover the instance space with extracted rules by successively generalizing training examples and testing these generalizations against the network.”
Rationalizing Neural Predictions (2016, background): “Attention based models offer another lens to the inner workings of neural models (Bahdanau et al., 2015; Cheng et al., 2016; Martins and Astudillo, 2016; Chen et al., 2015; Xu and Saenko, 2015; Yang et al., 2015).”
Explanation and Justification in Machine Learning : A Survey Or (2017, methodology): “Early work described approximations of the entire model [Thrun, 1995; Craven and Shavlik, 1999]; the disadvantage of these approaches is that for even moderately complex models, a good global approximation cannot generally be found.”
The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions (2016, background): “One solution to this problem is to provide human readable reasoning to justify or explain the answer, which has been a long-standing goal in Neural Networks (see [7, 28], for example).”
Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework (2017, background): “Much prior work has aimed to improve the interpretability of neural networks, focusing in particular on extracting rules from the activations of feed forward networks (Towell and Shavlik, 1993; Thrun, 1993; Fu, 1994; Thrun, 1995).”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “The mentioned exemplary rule-extraction methods DeepRED [152] and VIA [130], as well as decision tree extractors TRE- PAN [32], Concept Tree [111] and C4.5 [108] all provide global, rule-based output.”
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data (2016, background): “Finally, Socratic learning is also inspired by ideas similar to rule extraction [28, 6], where a complex model is simplified in terms of if-then statements.”