Forming sparse representations by local anti-Hebbian learning

By Peter Földiák. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “Many UL methods automatically and robustly generate distributed, sparse representations of input patterns (Falconbridge, Stamps, & Badcock, 2006; Földiák, 1990; Hinton & Ghahramani, 1997; Hochreiter & Schmidhuber, 1999; Hyvärinen, Hoyer, & Oja, 1999; Lewicki & Olshausen, 1998) throughwell-known…”
Barlow Twins: Self-Supervised Learning via Redundancy Reduction (2021, background): “, 2018; Schwartz & Simoncelli, 2001) for recent efforts), and has led to a number of algorithms for supervised and unsupervised learning (Ballé et al., 2017; Deco & Parra, 1997; Földiák, 1990; Linsker, 1988; Redlich, 1993a;b; Schmidhuber et al., 1996).”
A Survey of Inductive Biases for Factorial Representation-Learning (2016, methodology): “In [15], the author used direct inhibitory (“anti-Hebbian”) feedback connections between the representation factors, which has the eﬀect of of decreasing correlations between factors.”
The Utility of Sparse Representations for Control in Reinforcement Learning (2018, background): “…projecting data into a higher-dimensional space—include invariance (Goodfellow et al. 2009; Rifai et al. 2011); decorrelated features per instance (Földiák 1990); improved computational efﬁciency for updating weights in the predictor, as only weights corresponding to active features need to be…”
SPARSE CODING IN THE PRIMATE CORTEX (2002, unknown): “A simple unsupervised algorithm for learning such representations in a nonlinear network using local learning rules has been proposed (Foldiak, 1990) which uses Hebbian forward connections to detect nonaccidental features, an adaptive threshold to keep the activity ratio low, and anti-Hebbian,…”
Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks? (2017, methodology): “…been devoted to the self-organization of neural networks for solving unsupervised computational tasks using Hebbian and anti-Hebbian learning rules (F¨oldiak, 1990; F¨oldiak, 1989; Rubner and Tavan, 1989; Rubner and Schulten, 1990; Carlson, 1990; Plumbley, 1993b; Leen, 1991; Plumbley, 1993a;…”
Space and time in visual context (2007, background): “More mechanistic accounts include the suggestion that anti-Hebbian learning could be used to realize de-correlatio”
Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions (2013, unknown): “…componentanalysis(ICA),orsparselearning,mightberealizedinphys-iologicallyplausiblewaysbycombiningHebbianandanti-Hebbian learningruleswithintrinsicadaptationmechanismswithinindividual neurons(Fo¨ldia´k,1990;Falconbridgeetal.,2006;Gerhardetal.,2009).”
A Simple Hebbian/Anti-Hebbian Network Learns the Sparse, Independent Components of Natural Images (2006, methodology): “In this article, we show that two slightly modified versions of an early Hebbian/anti-Hebbian sparse coding algorithm (Földiák, 1990) are also capable of extracting the sparse, independent components of natural images.”
Independent component analysis of Gabor features for face recognition (2003, unknown): “2# % & (+*Ho+o (5) where and 7 denotetheFourierandinverseFouriertransform,respectively.”
Learning Invariance from Natural Images Inspired by Observations in the Primary Visual Cortex (2012, methodology): “The previous rule was developed on ideas of Hebbian learning (Oja, 1982), covariance learning (Sejnowski, 1977), and anti-Hebbian decorrelation (F¨oldi´ak, 1990) and lead to largely independent responses of V1 simple cells when trained on natural scenes.”
A theory of cortical responses (2005, background): “Horizontal interactions among the error units serve to decorrelate them (cf. Foldiak 1990), where the symmetric lateral connection strengths li hyperparameterize the covariances of the errors Si, which are the prior covariances for level iK1.”
Neuronal Synchrony in Complex-Valued Deep Networks (2013, methodology): “In this first experiment, we trained a DBM with one hidden layer (a restricted Boltzmann machine, Smolensky, 1986) on a version of the classic ‘bars problem’ (Földiák, 1990), where binary images are created by randomly drawing horizontal and vertical bars (Figure 3a).”
Binding via Reconstruction Clustering (2015, unknown): “Bars Introduced by F¨oldiak (1990)”
Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. (2003, methodology): “However, the way of arriving at that result will differ significantly; for example a PCA algorithm may be achieved via a supervised neural network (Bourlard and Kamp, 1988) or an unsupervised neural network (Foldiak, 1990; Kung and Diamantaras, 1990).”