iNNvestigate neural networks!

By Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grgoire Montavon, Wojciech Samek, K. Müller, Sven Dähne, Pieter-Jan Kindermans. Journal of machine learning research, 2018

380 citations recorded by the source corpus.

In recent years, deep neural networks have revolutionized many application domains of machine learning and are key components of many critical decision or predictive processes. Therefore, it is crucial that domain specialists can understand and analyze actions and pre- dictions, even of the most complex neural network architectures. Despite these arguments neural networks are often treated as black boxes. In the attempt to alleviate this short- coming many analysis methods were proposed, yet the lack of reference implementations often makes a systematic comparison between the methods a major effort. The presented library iNNvestigate addresses this by providing a common interface and out-of-the- box implementation for many analysis methods, including the reference implementation for PatternNet and PatternAttribution as well as for LRP-methods. To demonstrate the versatility of iNNvestigate, we provide an analysis of image classifications for variety of state-of-the-art neural network architectures.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d5d7b3c75c5a7bbc605236e7c722ecfb2cf8824e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploring chemical compound space with quantum-based machine learning (2019, methodology): “The iNNvestigate toolbox that allows to explain nonlinear learning methods such as deep learning is described in [161].”
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and Beyond (2020, methodology): “Both the DeepExplain [4] and iNNvestigate [3] toolboxes built on top of the popular Keras [26] package for Python with TensorFlow backend for explaining Deep Neural Network models, and thus provide support for both CPU and GPU hardware and convenient access for users of Keras models.”
Explainable Deep Learning Models in Medical Image Analysis (2020, methodology): “Most of the methods discussed here apart from the newer Deep Deep Learning Important FeaTures (LIFT) and Deep SHapley Additive exPlanations (SHAP) are implemented in the iNNvestigate toolbox [15].”
Explanations can be manipulated and geometry is to blame (2019, methodology): “These methods cover two classes of attribution methods, namely gradient-based and propagationbased explanations, and are frequently used in practice [26, 27].”
Explainable AI Methods - A Brief Overview (2020, methodology): “Furthermore, the GitHub library iNNvestigate provides a common interface and out-of-the-box implementation for many analysis methods, including LRP [2].”
Fairwashing Explanations with Off-Manifold Detergent (2020, methodology): “Furthermore, the chosen methods are widely used in practice (Kokhlikyan et al., 2019; Alber et al., 2019; Ancona et al., 2018).”
Towards Explainable Artificial Intelligence (2019, methodology): “The iNNvestigate toolbox [1] provides an efficient implementation for many of these propagation-based explanation methods.”
On the Privacy Risks of Model Explanations (2019, methodology): “Many feature-based explanation techniques are implemented in the innvestigate library2 [4] which we use in our experiments.”
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019, methodology): “Other approaches seek to identify interpretable representations in order to help understand how an algorithm works [1,11,29].”
Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “iNNvestigate [234] compares different methods for explainability, namely PatternNet, PatternAttribution and LRP.”
Human-grounded Evaluations of Explanation Methods for Text Classification (2019, methodology): “LRP (N) selected ‘armed bandit settings with’, ‘the Wasserstein distance’, and ‘derive policy gradients’ as evidence for the class PH.”
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications (2021, methodology): “Because highly predictive models are becoming increasingly complex both in terms of the number of parameters and structure of the model, XAI software must keep up with this increasing complexity [3], and at the same time, the human must also deal with explanations of increasingly subtler predictions”
Sequential Explanations with Mental Model-Based Policies (2020, methodology): “…within the ﬁeld of XAI and their demonstrated validity: prototypes (Kim et al., 2016; Gurumoorthy et al., 2017) and deep Taylor decomposition saliency maps (Montavon et al., 2017), implemented using the AIX360 (Arya et al., 2019) and the iNNvestigate (Alber et al., 2019) libraries, respectively.”