Analyzing Classifiers: Fisher Vectors and Deep Neural Networks

By Sebastian Bach, Alexander Binder, Grgoire Montavon, K. Müller, Wojciech Samek. 2015

205 citations recorded by the source corpus.

Fisher vector (FV) classifiers and Deep Neural Networks (DNNs) are popular and successful algorithms for solving image classification problems. However, both are generally considered 'black box' predictors as the non-linear transformations involved have so far prevented transparent and interpretable reasoning. Recently, a principled technique, Layer-wise Relevance Propagation (LRP), has been developed in order to better comprehend the inherent structured reasoning of complex nonlinear classification models such as Bag of Feature models or DNNs. In this paper we (1) extend the LRP framework also for Fisher vector classifiers and then use it as analysis tool to (2) quantify the importance of context for classification, (3) qualitatively compare DNNs against FV classifiers in terms of important image regions and (4) detect potential flaws and biases in data. All experiments are performed on the PASCAL VOC 2007 and ILSVRC 2012 data sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models (2017, methodology): “In the following we describe the LRP redistribution process for feed-forward neural networks, redistribution procedures have also been proposed for other popular models [5, 4, 20].”
Methods for interpreting and understanding deep neural networks (2017, methodology): “In practice, it is also essential to verify for a given task, that the high measured accuracy results from the use of a proper problem representation, and not from the exploitation of artifacts in the data [29, 46, 27].”
Controlling explanatory heatmap resolution and semantics via decomposition depth (2016, methodology): “For that reason [6, 7] uniformly distribute a local feature’s relevance score across all pixels within its receptive field to compute R p for each input pixel.”
Explaining nonlinear classification decisions with deep Taylor decomposition (2015, neutral): “The observation that more accurate predictions are supported by better resolved input patterns is also in line with other studies [51, 52].”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “flat rule Ri←j = 1 ∑ i 1 Rj - FV local descriptors [18,34] - Atari agents (bottom layers)”
Evaluating the Visualization of What a Deep Neural Network Has Learned (2015, neutral): “An approach, layer-wise relevance propagation (LRP), which is applicable to arbitrary types of neural unit activities (even if they are noncontinuous) and to general DNN architectures (and Fisher vector classifiers [27]) has been proposed in [28].”
Understanding intermediate layers using linear classifier probes (2016, neutral): “See also Lapuschkin et al. (2016).”
"What is relevant in a text document?": An interpretable machine learning approach (2016, methodology): “Note that the LRP framework can be applied to various models such as kernel support vector machines and deep neural networks [13, 18].”
Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers (2016, methodology): “This was done for neural networks, bag of word models [2, 11], and in a subsequent work [5], for Fisher vectors.”
Interpretable Deep Neural Networks for Single-Trial EEG Classification (2016, background): “, in computer vision [12, 13]) by making the prediction transparent.”
Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation (2016, methodology): “For instance, in [1] and [15] it has been applied to Bag-of-Words models and Fisher Vector / SVM classifiers, respectively.”
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019, methodology): “Layer-wise Relevance Propagation (LRP) [9,58] is a propagation-based explanation framework, which is applicable to general neural network structures, including deep neural networks [13], LSTMs [5,7], and Fisher Vector classifiers [44].”
Interpretable human action recognition in compressed domain (2017, methodology): “While these methods are limited in their range of applicability, general explanation techniques such as Layer-wise Relevance Propagation (LRP) [16] and Deep Taylor [17] have been recently introduced and applied to image, text and biomedical signal classification problems [18, 19, 20].”