Learning in linear neural networks: a survey

By Pierre Baldi, Kurt Hornik. 1995

Subfields: Model Architectures, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Statistical Learning Theory

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “…through training an NN first on shallow problems whose solutions may then generalize to deep problems, or through collapsing sequences of (non)linear operations into a single (non)linear operation (but see an analysis of non-trivial aspects of deep linear networks, Baldi & Hornik, 1995, Section B).”
Theoretical Analysis of Bayesian Matrix Factorization (2011, methodology): “With appropriate pre-whitening (Hyvärinen et al., 2001), reduced rank regression (Baldi and Hornik, 1995; Reinsel and Velu, 1998), canonical correlation analysis (Hotelling, 1936; Anderson, 1984), partial least-squares (Wold, 1966; Worsley et al., 1997; Rosipal and Krämer, 2006), and multi-task…”
Active Hebbian learning algorithm to train fuzzy cognitive maps (2004, methodology): “The steady-state weights deﬁne the learning process and the ANN model [28].”
Computational Intelligence in Wireless Sensor Networks: A Survey (2011, background): “A survey of NN learning methods, their properties and applications is presented in [32].”
Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview (2018, methodology): “This problem stems from interpreting principal component analysis (PCA) from an optimization perspective, which has a long history in the literature of (linear) neural networks and unsupervised learning; see for example [36–42].”
The dropout learning algorithm (2014, background): “As is often the case [8, 9], understanding dropout in linear networks is essential for understanding dropout in non-linear networks.”
Learning curves for stochastic gradient descent in linear feedforward networks (unknown): “We study the operation of exemplars of both on a feedforward linear perceptron, which has the advantage over the nonlinear case that the learning curves can be calculated exactly [8].”
Global analytic solution of fully-observed variational Bayesian matrix factorization (2013, result): “Finally, we applied VBMF to a reduced rank regression (RRR) (Reinsel and Velu, 1998) task, and show the results in Figure 7.”
Neural networks for classification: a survey (2000, background): “These issues are common to all applications of neural networks and some of them have been previously reviewed [4], [10], [29],”
Effect of Batch Learning in Multilayer Neural Networks (1998, methodology): “It is known that the MLE of the LNN model is analytically solvable ([5]), and it is practically absurd to use the steepest descent method.”