A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data

By Cengiz Pehlevan, Tao Hu, Dmitri "Mitya" Chklovskii. 2015

120 citations recorded by the source corpus.

Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis, by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost function, these rules are nonlocal and hence biologically implausible. At the same time, biologically plausible local rules have been postulated rather than derived from a principled cost function. Here, to bridge this gap, we derive a biologically plausible network for subspace learning on streaming data by minimizing a principled cost function. In a departure from previous work, where cost was quantified by the representation, or reconstruction, error, we adopt a multidimensional scaling cost function for streaming data. The resulting algorithm relies only on biologically plausible Hebbian and anti-Hebbian local learning rules. In a stochastic setting, synaptic weights converge to a stationary state, which projects the input data onto the principal subspace. If the data are generated by a nonstationary distribution, the network can track the principal subspace. Thus, our result makes a step toward an algorithmic theory of neural computation.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “New Hebbian rules have been recently proposed to minimize defined cost functions (Pehlevan et al., 2015; Bahroun et al., 2017).”
Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks? (2017, methodology): “te depends on the activity of only the two neurons the synapse connects. The only PSP objective known to yield a single-layer neural network with local learning rules is based on similarity matching (Pehlevan et al., 2015). This objective, borrowed from MultiDimensional Scaling (MDS), minimizes the ”
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (2024, methodology): “Several methods extend original Oja’s rule to top k principal components (Oja, 1989; Diamantaras & Kung, 1996; Pehlevan et al., 2015), and we focus on the subspace algorithm that updates weights as ∆ H = η yx ⊤ − yy ⊤ H , which enables weights to converge to a dominant principal sub-space (Yan et…”
Using Conceptors to Manage Neural Long-Term Memories for Temporal Patterns (2017, background): “Much research has been devoted to determine biologically plausible neural algorithms for the closely related task of computing a PCA or of finding some cost-optimal projection subspaces (Oja, 1982; Pehlevan et al., 2015).”