Opening black box Data Mining models using Sensitivity Analysis

By Paulo Cortez, Mark J. Embrechts. 2011

113 citations recorded by the source corpus.

There are several supervised learning Data Mining (DM) methods, such as Neural Networks (NN), Support Vector Machines (SVM) and ensembles, that often attain high quality predictions, although the obtained models are difficult to interpret by humans. In this paper, we open these black box DM models by using a novel visualization approach that is based on a Sensitivity Analysis (SA) method. In particular, we propose a Global SA (GSA), which extends the applicability of previous SA methods (e.g. to classification tasks), and several visualization techniques (e.g. variable effect characteristic curve), for assessing input relevance and effects on the model's responses. We show the GSA capabilities by conducting several experiments, using a NN ensemble and SVM model, in both synthetic and real-world datasets.

Subfields: Artificial Intelligence, Information Systems, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “In contrast, in [222] the authors build upon the existing SA (Sensitivity Analysis) to construct a Global SA which extends the applicability of the existing methods.”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “SA has been increasingly used in explaining ANN in general and DNN classification of images in particular [129] and [130].”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “From [16] VEC curve and histogram for the pH input feature (x-axis) and the respective high quality wine probability outcome (left of y-axis) and frequency (right of y-axis).”
Using data mining for bank direct marketing: an application of the CRISP-DM methodology (2011, methodology): “For instance, using a sensitivity analysis method (Cortez and Embrechts, 2011), we can characterize the SVM inputs influence in success.”
Using sensitivity analysis and visualization techniques to open black box data mining models (2013, methodology): “Regarding the use of visualization techniques, the majority of these methods address aspects related to the multidimensionality of data and the use of visualization for black box DM models is more scarce [21].”
A Survey on the Explainability of Supervised Machine Learning (2020, background): “GSA can consider multiple features at a time.”
Data-Based Models for the Prediction of Dam Behaviour: A Review and Some Methodological Considerations (2017, background): “More complex algorithms have been proposed in related fields to unveil the relevance of each input in NN models (see for example [67], [68] and [69]), which may be helpful in dam monitoring.”
Principles and Practice of Explainable Machine Learning (2020, methodology): “Di ﬀ erent from the above threads, in [18], the authors extend existing SA (Sensitivity Analysis) approaches in order to design a Global SA method.”
Explainable Artificial Intelligence: a Systematic Review (2020, background): “(a) Distill-and-Compare [148] (b) GSA [152] (c) GFA [160]”
A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations (2017, methodology): “For enhancing the interpretability of classiﬁer predictions, Cortez and Embrechts [9] use a sensitivity analysis approach for letting users understand the e ﬀ ects of variation of input values on model outputs.”
A Framework for Considering Comprehensibility in Modeling (2016, unknown): “For example, Tzeng and Ma 20 provide visualizations of neural network architectures (internals) whereas Cortez and Embrechts 21 derive visualizable representations of the relationships (sensitivities) between model inputs and outputs.”