Using Sampling and Queries to Extract Rules from Trained Neural Networks

By Mark W. Craven, Jude W. Shavlik. 1994

Subfields: Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “Pedagogical approach Introduced by [9] and named by [3] the pedagogical approach involves “viewing rule extraction as a learning task where the target concept is the function computed by the network and the input features are simply the network’s input features.”
A Survey of Methods for Explaining Black Box Models (2018, background): “DT TE TAB X Conj Rules [19] Craven et al.”
Rule Extraction Algorithm for Deep Neural Networks: A Review (2016, background): “Craven and Shavlik [23] have proposed a pedagogical algorithm called TREPAN.”
Natural Language Processing and Machine Learning (2016, unknown): “Conjunctive rule-extraction algorithm: Carven & Shavlik’s [10] approach involves viewing rule-extraction as a learning task where the target concept is the function computed by the network and the input features are simply the network's input features.”
On The Stability of Interpretable Models (2018, methodology): “We restrict here to class balancing and random sampling methods, because they are widely adopted in black-box explanation approaches [5], [32].”
Explaining Classifications For Individual Instances (2008, methodology): “Techniques for extracting explicit (if-then) rules from black box models (such as ANN) are described in [9–12].”
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2023, methodology): “An approach was provided in the study by Craven et al. [96] in 1994 that entailed the iterative creation and updating of a set of rules applied across the input and output classes of the ANN until all of the target classes have been processed.”
Neural network explanation using inversion (2007, background): “TREPAN of Craven and Shavlik (1994) extracts rules in the form of a decision tree.”
Extracting symbolic rules from trained neural network ensembles (2003, background): “Craven and Shavlik [7] regard rule extraction as a learning task and extract conjunctive rules with the help of two oracles.”
Symbolic Interpretation of Artificial Neural Networks (1999, background): “BRAINNE [33], Rule-extraction-aslearning [ 6 ], and DEDEC [44] are other examples of extracting rules by investigating the input-output mapping of a trained network.”