Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells

By Mathias Franzius, Henning Sprekeler, Laurenz Wiskott. 2007

242 citations recorded by the source corpus.

We present a model for the self-organized formation of place cells, head-direction cells, and spatial-view cells in the hippocampal formation based on unsupervised learning on quasi-natural visual stimuli. The model comprises a hierarchy of Slow Feature Analysis (SFA) nodes, which were recently shown to reproduce many properties of complex cells in the early visual system [1]. The system extracts a distributed grid-like representation of position and orientation, which is transcoded into a localized place-field, head-direction, or view representation, by sparse coding. The type of cells that develops depends solely on the relevant input statistics, i.e., the movement pattern of the simulated animal. The numerical simulations are complemented by a mathematical analysis that allows us to accurately predict the output of the top SFA layer.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution (2018, background): “via non-Cartesian representations like grid cells [1, 6, 3]? A complete answer to this question is beyond the scope of this paper, but encouraging preliminary evidence shows that swapping Conv for CoordConv can improve a diverse set of models — including ResNet-50, Faster R-CNN, GANs, VAEs, and RL m”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “…within certain models gives rise to aspects of the visual system including complex cells (Körding et al., 2004), speciﬁc properties of visual invariance (Isik et al., 2012), and even other representational properties such as the existence of place cells (Wyss et al., 2006; Franzius et al., 2007).”
Invariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet (2012, background): “, 2006), and slowness as an asset in learning spatial representations has also been investigated by others (Wiskott and Sejnowski, 2002; Wiskott, 2003; Franzius et al., 2007).”
Spectral Inference Networks: Unifying Deep and Spectral Learning (2018, methodology): “…for Laplacian eigenmaps (Sprekeler, 2011), and it has been shown that optimizing for the slowness of features in navigation can also lead to the emergence of units whose response properties mimic grid cells in the entorhinal cortex of rodents (Wyss et al., 2006; Franzius et al., 2007).”
Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams (2012, methodology): “It is possible to include clipping Franzius et al. (2007) in BSFA, so that the effect of the outliers that have different variance statistics compared to the signal can be overcome.”
Invariant Object Recognition with Slow Feature Analysis (2008, methodology): “The configurations for the training sequences were generated as a random walk procedure like in [ 2 ].”
Learning and disrupting invariance in visual recognition (2011, methodology): “Temporal association learning rules provide a plausible way to learn transformation invariance through natural visual experience [1, 7, 8, 9, 10, 11, 12, 13, 14].”
Modular Toolkit for Data Processing (MDP): A Python Data Processing Framework (2008, unknown): “MDP has been used to implement a model of the visual system of a virtual rat moving around in a virtual environment (Franzius et al., 2007), to perform pattern recognition (Franzius et al., 2008) and handwritten digit recognition (Berkes, 2006), to analyze intra-cerebral array-recorded…”
Construction of approximation spaces for reinforcement learning (2013, background): “Although proven to work in complex tasks (Franzius et al., 2007), this approach involves a multitude of hyper-parameters and no easy way to counteract inevitable over-fitting is known so far.”
How to solve classification and regression problems on high-dimensional data with a supervised extension of slow feature analysis (2013, methodology): “The concise formulation of the SFA optimization problem also permits an extended mathematical treatment so that its properties are well understood analytically (Wiskott, 2003; Franzius et al., 2007; Sprekeler and Wiskott, 2011).”
Predictable Feature Analysis (2013, methodology): “For instance, tasks like the self-organization of complex-cell receptive fields, the recognition of whole objects invariant to spatial transformations, the self-organization of place-cells, extraction of driving forces, or nonlinear blind source separation were successfully performed on the basis of”
An extension of slow feature analysis for nonlinear blind source separation (2014, methodology): “Here, we extend a previously developed mathematical analysis of SFA (Franzius et al., 2007) to the case where the input data are generated from a set of statistically independent sources.”
Slow Feature Analysis for Human Action Recognition (2012, methodology): “SFA has been successfully applied to learning the receptive field of visual cortical neuron [2], [3].”
Reinforcement Learning on Slow Features of High-Dimensional Input Streams (2010, background): “This has led to place-cell like behavior in [14] and might be beneficial for subsequent reward-based learning.”
Design Principles of the Hippocampal Cognitive Map (2014, methodology): “Applying hierarchical slow feature analysis to streams of simulated visual inputs produces feature representations that resemble hippocampal receptive fields [12].”
Spectral Inference Networks: Unifying Spectral Methods With Deep Learning (2018, methodology): “SFA is equivalent to function approximation for Laplacian eigenmaps [30], and it has been shown that optimizing for the slowness of features in navigation can also lead to the emergence of grid cells [37, 14].”
Autonomous Learning of State Representations for Control: An Emerging Field Aims to Autonomously Learn State Representations for Reinforcement Learning Agents from Their Real-World Sensor Observations (2015, background): “this feature space [13], which explains the good perfor-”