The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

By Alan B. Tickle, Robert Andrews, Mostefa Golea, Joachim Diederich. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Opening black box Data Mining models using Sensitivity Analysis (2011, background): “The extraction of rules is the most popular solution [7][8][9].”
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions (2018, background): “The increasing popularity of machine learning models as decision making tools presents novel challenges for explanation, as it may not be as straightforward to extract the embedded ‘knowledge’ behind their outputs [73].”
Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018, background): “Research stemming from the early days of machine learning in expert systems attempts to make algorithmic logics more transparent [75, 81, 54, 66], or to better understand how users work with inductive systems and build mental models of them [60, 48, 77].”
Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems (2012, background): “It is the process of developing natural language like syntax that describes the behaviour of a neural network [12].”
Symbolic Interpretation of Artificial Neural Networks (1999, background): “EVERAL researchers have investigated the design of hybrid systems that combine expert and connectionist sub-systems [40], [47], [7], [12], [11], [20], [48].”
Ethical Decision Making during Automated Vehicle Crashes (2014, background): “This process essentially translates a neural network's internal knowledge into a set of symbolic rules, which can then be expressed as natural language (55) .”
Neural networks for classification: a survey (2000, background): “Numerous insights have also been gained into the neural networks in performing classification as well as other tasks [23], [169].”
Mortality assessment in intensive care units via adverse events using artificial neural networks (2006, background): “Currently, there are two main approaches [27]: the use of decompositional and pedagogical techniques.”
Enslaving the Algorithm: From a “Right to an Explanation” to a “Right to Better Decisions”? (2018, unknown): “It is sometimes possible however to devise explanations of systems which do not need access to the proprietary innards of an ML model using “pedagogical” or “model-agnostic” methods [10, 12].”
Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review (2005, background): “typically not designed to be easy to analyse) are described by Tickle et al. (1998) as more attractive techniques.”
Explaining models: an empirical study of how explanations impact fairness judgment (2019, background): “Given the complexity of current ML models, explanations are often pedagogical [34], meaning that they reveal information about how the model works without faithfully representing the algorithms.”
Natural Language Processing and Machine Learning (2016, unknown): “the feature is left undetermined), otherwise it is put back on the rule) [2] [3].”