Simplified neuron model as a principal component analyzer

By Erkki Oja. 1982

Subfields: Representation & Self-Supervised Learning, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Neuroscience

Citations in the corpus

PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization (2019, neutral): “…M t and M t − 1 are stochastic approximations of the full gradient, we can expect that ≈ The result is akin to Oja’s algorithm for stochastic power iteration (Oja, 1982), and hence could result in an improved approximation quality. since the function is smooth and we only take small update steps.”
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD (2018, methodology): “(4)Oja’s algorithm [28] is itself an SGD variant of power method to find approximate eigenvectors.”
View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation (2016, positive): “In the following we consider as an example a simple Hebbian learning scheme called Oja’s rule [37, 38].”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Variations have been proposed to include, e.g. terms to achieve stability (Oja, 1982; Bienenstock et al., 1982) and various constraints (Miller and Mackay, 1994), or more advanced update dynamics such as dual weights for fast and slow decay (Levy and Bairaktaris, 1995; Soltoggio, 2015).”
The temporal paradox of Hebbian learning and homeostatic plasticity (2017, background): “While the observed weightdependence is reminiscent of Oja’s rule [8], as strong synapses weaken, it is not identical be-”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “The method of Oja (1982) initializes a random orthonormal matrix U (1) ∈ Rn×r, updates U (k+1) ← orth ( U (k) + γ(k)YkY > k U (k) ) (30) at iteration k with stepsize γ(k).”
Simple, Efficient, and Neural Algorithms for Sparse Coding (2015, methodology): “(a) It is standard to compute the inner products (cid:104) y, u (cid:105) and (cid:104) y, v (cid:105) using neurons, and even the top singular vector can be computed using the classic Oja’s Rule Oja (1982) in an online manner where each sample y is received sequentially.”
Natasha 2: Faster Non-Convex Optimization Than SGD (2017, methodology): “We propose to use Oja’s algorithm [37] to approximate power method.”
Models of Metaplasticity: A Review of Concepts (2015, background): “While this instability can be avoided by artificially imposing hard boundaries onto the synaptic weights, several learning models came with intrinsic mechanisms regulating the synaptic efficacies (Bienenstock et al., 1982; Oja, 1982) in order to solve this issue in a less fine-tuned manner.”
Robust principal component analysis for computer vision (2001, background): “The above techniques are of limited application in computer vision problems as they reject entire images as outliers.”
On the methods for reactivation and replay analysis (2020, methodology): “This is done by decomposing the correlation matrix of the binned spike trains of the reference epoch into its eigenvectors (i.e. PCA) and determining the number of its eigenvalues (λ) that are greater than the theoretical upper bound of the eigenvalue spectrum (the Marchenko–Pastur bound, λmax).”
Phenomenological models of synaptic plasticity based on spike timing (2008, background): “(i) sensitivity to correlations between pre- and postsynaptic neurons (Hebb 1949) in order to respond to correlations in the input (Oja 1982).”
Evolutionary and Computational Advantages of Neuromodulated Plasticity (2008, methodology): “Alternative models like the Oja rule (Oja, 1982) and the BCM rule (Bienenstock et al.”
Fast and Simple PCA via Convex Optimization (2015, methodology): “Recently Shamir [24, 25] made headway in the non-convex spectral world by suggesting a stochastic optimization method for the problem, that is based Oja’s randomized Power method [19, 4] with an application of the variance reduction principle demonstrated in [14].”
Stable Hebbian Learning from Spike Timing-Dependent Plasticity (2000, background): “An effective way to store memories is to introduce correlation among inputs ( Oja, 1982; Song et al., 2000).”