A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images.

By Aapo Hyvärinen, Patrik O. Hoyer. Vision Research, 2001

301 citations recorded by the source corpus.

Subfields: Biophysics, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/7c032d555a9d7096f7bb88441f10e33d3302d5be

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Cognitive Neuroscience

Citations in the corpus

A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex (2005, unknown): “This is a reasonable assumption considering recent theoretical work that has shown that neurons in primary visual cortex are tuned to the statistics of natural images [Olshausen and Field, 1996; Hyvärinen and Hoyer, 2001].”
On the Analysis and Interpretation of Inhomogeneous Quadratic Forms as Receptive Fields (2006, methodology): “Quadratic forms were also used in various theoretical papers, either explicitly (Hashimoto, 2003; Bartsch and Obermayer, 2003) or implicitly in the form of neural networks (Hyvärinen and Hoyer, 2000; Hyvärinen and Hoyer, 2001; Körding et al., 2004).”
Learning Invariance from Natural Images Inspired by Observations in the Primary Visual Cortex (2012, background): “This higher amount is consistent with the previously reported results that residual dependencies in input can be enough for learning invariance (Hyv¨arinen & Hoyer, 2000, 2001).”
Learning invariant features through topographic filter maps (2009, background): “Highly-active simple cells inhibit other cell s at neighboring locations and orientations (similarly to lo cal normalization and/or quantization), while complex cel ls spatially pool the rectified outputs of complex cells, so as to create a local invariance to small shifts (like the poolin g operati”
Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation (2006, background): “topographic maps from natural image patches ( Hyvarinen and Hoyer, 2001 ) are also”
Estimation of Non-Normalized Statistical Models by Score Matching (2005, methodology): “Other recent work in image modelling also includes non-normalized models (Hyvärinen and Hoyer, 2001; Teh et al., 2003).”
Computational Maps in the Visual Cortex (2005, background): “Although a few such models have been proposed (Barrow and Bray 1992; Hyvärinen and Hoyer 2001; Shouval et al. 1997; Weber 2001), most models do not include these requisite processing stages.”
Efficient Learning of Sparse Representations with an Energy-Based Model (2006, methodology): “Popular methods such as Wavelet dec omposition, PCA, Kernel-PCA, NonNegative Matrix Factorization [1], and ICA produce compact representations with somewhat uncorrelated (or independent) components [2].”
THE CO-INFORMATION LATTICE (2003, background): “(16), or, equivalently, maximise the likelihood, by adjusting the weights with ∆W ∝ − (∇ W H G (x)) W T W giving a stochastic gradient learning rule: ∆W ∝ I + f (u)u T W − (∇ W Z G (x)) W T W (18) The familiar first term is as in ICA, except that now the non-linearities (the score functions), crucia”
Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields (2010, background): “Hyvarinen and Hoyer have proposed to use group sparsity constraints in a sparse reconstruction framework to force similar filters to gather within groups (Hyvarinen and Hoyer, 2001).”
Signal recovery from Pooling Representations (2013, unknown): “In the trained system, the columns of the weight matrix corresponding to nodes grouped together often exhibit similar characteristics, and code for perturbations of a template (Kavukcuoglu et al., 2009; Hyvärinen and Hoyer, 2001).”
Synergy, Redundancy, and Independence in Population Codes (2003, background): “Independent component analysis can, in part, take these correlations into account, producing components that approximate activity independence and resemble V1 receptive fields (Bell and Sejnowski, 1997; Hyvarinen and Hoyer, 2001).”
Recursive ICA (2006, methodology): “Several extensions of the linear ICA algorithm [5, 6, 7, 8] have been proposed to reduce such residual nonlinear redundancy, with an explicit or implicit aim of explaining higher perceptual layers, such as complex cells in V1.”
A Two-Layer Model of Natural Stimuli Estimated with Score Matching (2010, methodology): “In Figure 4 we analyze the complex cell properties of the higher-order outputs for the different models, following Hyvärinen and Hoyer (2001).”
Learning Invariant Feature Hierarchies (2012, methodology): “This can be done with the idea of group sparsity [38] [49] [50], which is reminiscent of sub-space ICA [51] and Product of Experts [52]. the main idea is to replace the sparsity penalty in the PSD energy function by a group sparsity penalty: where g is an index over groups of components of Z (possib”