Extracting comprehensible models from trained neural networks

By Mark W. Craven, Jude W. Shavlik. 1996

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “• Comprehensibility : When conceived for ML models, comprehensi-bility refers to the ability of a learning algorithm to represent its learned knowledge in a human understandable fashion [19–21] .”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Another way to extract rules using sampling methods [32], [33].”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “The term XAI dates from∼2016, and grew out of work on function approximation; i.e., explaining a black box model by a simpler model (e.g., Craven and Shavlik, 1995; Craven, 1996), or explaining a black box using local approximations.”
Rule Extraction Algorithm for Deep Neural Networks: A Review (2016, background): “According to [14] rule extraction is defined as "…given a trained neural network and the data on which it was trained, produce a description of the network's hypothesis that is comprehensible yet closely approximates the network's predictive behavior."”
Comprehensible credit scoring models using rule extraction from support vector machines (2007, neutral): “Trepan Trepan was first introduced in [6,7].”
RISE: Randomized Input Sampling for Explanation of Black-box Models (2018, methodology): “) with simple, interpretable models such as decision rules or sparse linear models [3, 28].”
Predicting glass transition temperatures using neural networks (2018, methodology): “are studying methods to extract information from its internal parameters that could be used to build simpler and easily interpretable models, such as decision trees [66] or physical models (Occam's razor [67]).”
A Survey of Methods for Scaling Up Inductive Algorithms (1999, methodology): “If these operations can be cast as SQL requests for statistics (Agrawal and Shim, 1995, 1996), a data mining program can avoid massive data uploads and problems due to main memory restrictions, and can take advantage of fast database machines optimized for query processing.”
Interacting meaningfully with machine learning systems: Three experiments (2009, background): “This approach balances the issues of fidelity and comprehensibility (Craven, 1996; van de Merckt and Decaestecker, 1995).”
KDRuleEx: A Novel Approach for Enhancing User Comprehensibility Using Rule Extraction (2012, methodology): “Output: DTable: Two dimensional matrix to represent a Decision Table in which First Column is used for Features, First Row is used to store Decision and Second Row is used for Rule Length.”
Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring (2008, methodology): “The search process is initialized by ﬁrst selecting the best binary split at a given node based on the information gain criteria ([17] (or gain ratio according to [18]).”
Constructive neural-network learning algorithms for pattern classification (2000, methodology): “The question now is whether we can use some of the existing strategies (see, for example, [9]) or design suitable new methods for extracting the learned knowledge from a trained constructive network.”
The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks (1998, methodology): “The TREPAN algorithm described by Craven and Shavlik [7] and Craven [5] (see Table IV), is a general purpose algorithm for extracting a decision tree from any learned model, (ANN or symbolic).”
Occam's Two Razors: The Sharp and the Blunt (1998, background): “Systems that first induce the most accurate model they can, and then extract from it a more comprehensible model of variable complexity (e.g., (Craven 1996; Domingos 1997a)) seem a promising avenue.”
Evolutionary Fuzzy Systems for Explainable Artificial Intelligence: Why, When, What for, and Where to? (2019, background): “may be inspected and understood by humans [15], [18].”
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning (2018, methodology): “Another way to extract rules using sampling methods [32], [33].”