Extracting Refined Rules from Knowledge-Based Neural Networks

By Geoffrey G. Towell, Jude W. Shavlik. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Other notable rule-extraction techniques include the MofN algorithm [36], which tries to find rules that explain single neurons by clustering and ignoring insignificant neurons.”
Stealing Machine Learning Models via Prediction APIs (2016, methodology): “In a non-malicious setting, improper model extraction techniques have been applied for interpreting [2, 19, 52] and compressing [16, 27] complex neural networks.”
Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, methodology): “Traditional Interpretability Interpretable Models: [106], [103], [64], [102], [65], [105], [50], [13], [40] Causal Interpretability Model-based: [77], [38], [12], [8], [112], [81], [7], Example-based: [35], [45], [39], [101], [76], [86], [73], [61], [60], [33], [34] Post-hoc: [47], [89], [66], [93],”
Or's of And's for Interpretable Classification, with Application to Context-Aware Recommender Systems (2015, background): “Or’s of and’s models are also a special case of another form of interpretable model called M of-N rules [12,15,35,42,44], inparticular when M=1.”
Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework (2017, background): “Much prior work has aimed to improve the interpretability of neural networks, focusing in particular on extracting rules from the activations of feed forward networks (Towell and Shavlik, 1993; Thrun, 1993; Fu, 1994; Thrun, 1995).”
Explaining prediction models and individual predictions with feature contributions (2014, unknown): “Complex models, such as artificial neural networks [14, 15, 16, 17, 18] and SVM (Support Vector Machines) [19, 20, 10, 21, 22, 23, 24, 25] received most attention, because they are often very successful but non-transparent.”
Efficiently Explaining Decisions of Probabilistic RBF Classification Networks (2011, methodology): “Techniques for extracting explicit (if-then) rules from black box models (such as ANN) are described in [11, 7].”
An Efficient Explanation of Individual Classifications using Game Theory (2010, background): “The explanation and interpretation of artificial neural networks, which are arguably one of the least transparent models, has also received a lot of attention, especially in the form of rule extraction (Towell and Shavlik, 1993; Andrews et al., 1995; Nayak, 2009).”
Computational intelligence methods for rule-based data understanding (2004, methodology): “In the Subset algorithm of Towell and Shavlik [74], inputs with largest weights are analyzed first, and if they are sufficient to activate the hidden node of the network irrespective of the values on other inputs, a new rule is recorded.”
Non-iterative Knowledge Fusion in Deep Convolutional Neural Networks (2018, background): “The research devoted to the knowledge extraction and insertion received recently some attention [23, 24].”
Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review (2005, methodology): “Based on previous work, such as Towell & Shavlik (1993), four sub-aspects of rule quality were suggested in the ADT taxonomy; • rule accuracy , i.e. the ability of the rules to generalise to unseen examples, • rule ﬁdelity , i.e. how well the rules mimic the behaviour of the RNN, • rule consistency…”
Learning Causal State Representations of Partially Observable Environments (2019, methodology): “The idea of extracting the implicit knowledge of a neural network [37, 11, 28, 15] is not novel and is rooted in early attempts to merge traditional rule-based methods with machine learning.”