Neural Networks, Principal Components, and Subspaces

By Erkki Oja. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “Many papers of the previous millennium, however, were about unsupervised learning (UL) without a teacher (e.g., Atick, Li, & Redlich, 1992; Baldi & Hornik, 1989; Barlow, Kaushal, & Mitchison, 1989; Barrow, 1987; Deco & Parra, 1997; Field, 1987; Földiák, 1990; Földiák & Young, 1995; Grossberg, 1976a,”
Learning Factorial Codes by Predictability Minimization (1992, methodology): “convey the same information about the input. Setting , j = 0 means to maximize information locally for each unit while at the same time trying to force each unit to focus on different pieces of information from the environment. Unlike with autoassociators, there is I Z O global invertibility term. N”
Unsupervised Neural Learning on Lie Group (2002, background): “44 It arises from the gradient-based optimization of the criterion: where L is a symmetric matrix c containing the Lagrange multipliers for the constraint M T M = m 20 I q .”
Learning in linear neural networks: a survey (1995, methodology): “) The former choice, S = yy0, gives A = (yx0 yy0A); (26) which is called the \subspace" algorithm in Oja [38].”
Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks? (2017, background): “While Hebbian learning is famously observed in neural circuits (Bliss and Lømo, (Oja, 1989) and the GHA (Sanger, 1989).”
Image processing with neural networks - a review (2002, methodology): “A well-known featureextraction ANN is Oja’s neural implementation of a one-dimensional Principal Component Analysis (PCA) [61], later extended to multiple dimensions [62].”
Streaming Pattern Discovery in Multiple Time-Series (2005, methodology): “We experimented with several alternatives [30, 15] and found this particular method to have the best properties for our setting: it is very efficient in terms of computational and memory requirements, while converging quickly, with no special parameters to tune.”
Neural Network Approaches to Image Compression (1995, unknown): “This rule has been shown to converge to the largest principal component of the input x [28], [29], [30], [31].”
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (2024, methodology): “This work mainly focuses on Hebbian learning based on neuronal spiking activities, which can be shown to optimize synaptic weights for principal component analysis (Oja, 1982; 1989).”