Kernel Analysis of Deep Networks

By Grgoire Montavon, Mikio L. Braun, K. Müller. 2011

143 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

A Closer Look at Memorization in Deep Networks (2017, methodology): “Thus, our analysis builds on the work of Zhang et al. (2017) and further investigates the role of memorization in DNNs.”
Explaining nonlinear classification decisions with deep Taylor decomposition (2015, methodology): “Some methods seek to provide a global understanding of the trained model, by measuring important characteristics of it, such as the noise and relevant dimensionality of its feature space(s) [28, 29, 30], its invariance to certain”
Convergent Learning: Do different neural networks learn the same representations? (2015, background): “…studies have focused on understanding deep neural networks from both a theoretical perspective (Arora et al., 2014; Neyshabur & Panigrahy, 2013; Montavon et al., 2011; Paul & Venkatasubramanian, 2014; Goodfellow et al., 2014) and from an empirical perspective (Erhan et al., 2009; Eigen et al.,…”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, background): “It can be shown that from layer to layer, networks compress information and enhance the signal-to-noise ratio (see [109,110]).”
Machine learning of molecular electronic properties in chemical compound space (2013, positive): “The remarkable predictive power of the ML model can be rationalized by (i) the deep layered nature of the NN model that permits us to progressively extract the relevant problem subspace from the input representation and gain predictive accuracy [59, 60]; (ii) inclusion of random Coulomb matrices for”
Evaluating the Visualization of What a Deep Neural Network Has Learned (2015, methodology): “Several methods have been developed to understand what a DNN has learned [17], [18].”
Understanding intermediate layers using linear classifier probes (2016, background): “While our proposed idea shares commonalities with Montavon et al. (2011), our analysis is very different. ar X iv :1 61 0.”
Flattened Convolutional Neural Networks for Feedforward Acceleration (2014, background): “As the difﬁculty of classiﬁcation problem increases, the more number of leading components is required to solve the problem (Montavon et al., 2011).”
Layer-Wise Relevance Propagation for Deep Neural Network Architectures (2016, background): “Recently several methods have been proposed for analyzing what a deep neural network has learned [9, 17, 18].”
A Toolbox for Representational Similarity Analysis (2014, methodology): “Another relevant recent technique is kernel analysis, which can reveal the complexity of categorical boundaries in a representation [44–45].”
A Survey of Modern Questions and Challenges in Feature Extraction (2015, background): “The works of (Montavon et al., 2011; Bach et al., 2015) analyzed deep network with the idea of relevance decomposition - that is, determining which inputs/pixels are important for an image to be classified as what type of objects.”
SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement (2017, background): “Facets of this key question have been explored empirically, particularly for image models, in (1; 2; 8; 10; 11; 12; 13; 16; 17).”
Layer-wise training of deep networks using kernel similarity (2017, methodology): “Interestingly, it has been observed that as deeper and deeper kernels are built, simpler and more accurate representations of the learning problem are obtained [11].”
A unified deep artificial neural network approach to partial differential equations in complex geometries (2017, background): “As a more speculative note on future research we recall that is a well-know fact, in for example image analysis, that the different layers of deep ANN are responsible for detecting different features [9, 27, 28].”