Slow feature analysis yields a rich repertoire of complex cell properties.

By Pietro Berkes, Laurenz Wiskott. 2005

324 citations recorded by the source corpus.

In this study we investigate temporal slowness as a learning principle for receptive fields using slow feature analysis, a new algorithm to determine functions that extract slowly varying signals from the input data. We find a good qualitative and quantitative match between the set of learned functions trained on image sequences and the population of complex cells in the primary visual cortex (V1). The functions show many properties found also experimentally in complex cells, such as direction selectivity, non-orthogonal inhibition, end-inhibition, and side-inhibition. Our results demonstrate that a single unsupervised learning principle can account for such a rich repertoire of receptive field properties.

Subfields: Biophysics, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Cognitive Neuroscience

Citations in the corpus

Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “However, specialized mechanisms of slow feature analysis can be implemented that would yield invariance to complex input transformations such as view invariance (e.g., Berkes and Wiskott, 2005; Einhäuser et al., 2005).”
Manitest: Are classifiers really invariant? (2015, background): “06535v1 Due to the huge research efforts that have been recently deployed in computer vision and machine learning, the state-of-the-art image classiﬁcation systems are now reaching performances that are close to those of the human visual system in terms of accuracy on some datasets [18, 33].”
Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields (2010, methodology): “In the so-called slow feature analysis method (Wiskott and Sejnowski, 2002; Berkes and Wiskott, 2005; Bergstra and Bengio, 2009), this is achieved by penalizing the rate of change of unit activations as the input varies.”
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video (2015, background): “example, altering the natural temporal contiguity of visua l stimuli hinders translation invariance in the inferior tem poral cortex [27], and functions learned to preserve temporal coherence share behaviors observed in complex cells of the primary visual cortex [4].”
Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells (2007, methodology): “found by means of SFA [1,20].”
On the Analysis and Interpretation of Inhomogeneous Quadratic Forms as Receptive Fields (2006, methodology): “To illustrate the analysis techniques presented here we make use of the quadratic forms presented in (Berkes and Wiskott, 2002) in the context of a theoretical model of self–organization of complex–cell receptive ﬁelds in the primary visual cortex (see also Berkes and Wiskott, 2003).”
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives (2012, background): “A good overview can be found in Hurri and Hyv ¨ arinen (2003); Berkes and Wiskott (2005).”
Learning viewpoint invariant object representations using a temporal coherence principle (2005, methodology): “2004), ‘slow feature analysis’ (Berkes and Wiskott 2003) and a physiologically inspired learning rule (Einhäuser et al.”
How to solve classification and regression problems on high-dimensional data with a supervised extension of slow feature analysis (2013, methodology): “Berkes and Wiskott (2005) subsequently used it for learning complex-cell receptive fields and Franzius et al. (2007) for place cells in the hippocampus.”
Slow Feature Analysis for Human Action Recognition (2012, methodology): “SFA has been successfully applied to learning the receptive field of visual cortical neuron [2], [3].”
Learning Invariance from Natural Images Inspired by Observations in the Primary Visual Cortex (2012, methodology): “SFA also leads to orientation selectivity, but frequently for conjunct orientations (mean orientation bandwidth is 40.8 ◦ ).”
Invariant Object Recognition with Slow Feature Analysis (2008, methodology): “Slow Feature Analysis solves the following learning task: Given a multidimensional input signal we want to find instantaneous scalar input-output functions that generate output signals that vary as slowly as possible but still carry significant information.”
An extension of slow feature analysis for nonlinear blind source separation (2014, methodology): “If F is finite-dimensional, the problem can be solved efficiently by the SFA algorithm (Wiskott and Sejnowski, 2002; Berkes and Wiskott, 2005).”
Measuring Invariances in Deep Networks (2009, methodology): “6 Natural video test While the grating-based invariance test allows us to systematically vary the parameters used to generate the images, it shares the difficulty faced by a number of other methods for quantifying invariance that are based on synthetic (or nearly synthetic) data [19, 20, 21]: it is ”
Construction of approximation spaces for reinforcement learning (2013, methodology): “Although formulated as a linear algorithm, SFA was originally intended to be applied on the space of polynomials like quadratic (Wiskott and Sejnowski, 2002) or cubic (Berkes and Wiskott, 2005).”
Toward a unified theory of efficient, predictive, and sparse coding (2017, background): “In this case (and subject to some additional assumptions, such as Gaussian stimulus statistics and instantaneous encoding filters), predictive coding is formally equivalent to slow feature analysis (39).”
Tiled convolutional neural networks (2010, unknown): “By visualizi ng [28, 29] the range of optimal stimulus that activate each pooling unit in a Tiled CNN, we found units that were scale and rotationally invariant.”
Slow Feature Analysis for Change Detection in Multispectral Imagery (2014, background): “…and constraints (3) and (4) can be rewritten as We can integrate constraint (3) into objective (1), and the optimization problem can be rewritten as [44], [46] Δ( z j ) = ˙ z 2 j This optimization problem can be solved corresponding to the generalized eigenvalue problem AW = BWΛ (10) where A = (cid”