Backpropagation is Sensitive to Initial Conditions

By John F. Kolen, Jordan B. Pollack. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos (2019, background): “While we have yet to computationally conﬁrm our deep hypothesis, several works have investigated chaos in neural networks (Kolen & Pollack, 1991; Sompolinsky et al., 1988), although we believe we are the ﬁrst to suggest that chaos may cause gradients to degenerate when computed using…”
Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks (2017, background): “This results in quite diverse classifiers with different final weights [21].”
Multiple Classifier Systems (2000, methodology): “If the algorithm is applied to the same training examples but with different initial weights, the resulting classifier can be quite different ( Kolen & Pollack, 1991 ).”
Sequential Constant Size Compressors for Reinforcement Learning (2011, unknown): “[15, 12] for two decades.”
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms (1998, unknown): “Theresult-inglearnednetworkdependscriticallyontherandomstartingstate(Kolen & Pollack, 1991).”
The Induction of Dynamical Recognizers (1991, background): “But chaotic behavior has shown up repeatedly in studies of neural networks (Derrida & Meir, 1988; Huberman & Hogg, 1987; Kolen & Pollack, 1990; Kurten, 1987; van der Maas et al., 1990), and a few scientists have begun to explore how this dynamical complexity could be exploited for useful purposes,…”
Continuous latent variable models for dimensionality reduction and sequential data reconstruction (2001, background): “Backpropagation, for example, is sensitive to initial conditions (Kolen and Pollack, 1990).”
Ensemble Methods in Machine Learning (2007, methodology): “If the algorithm is applied to the same training examples but with diierent initial weights, the resulting classiier can be quite diierent (Kolen & Pollack, 1991).”
Learning to Adapt to Changing Environments in Evolving Neural Networks (1996, background): “…the case of our creatures we should consider that initial conditions (e.g., initial weights) can determine the course of backpropagation learning (Kolen and Pollack, 1990) and evolution can select for good initial weights for learning during life in nonecological neural networks (Belew et al.,…”
Deep forest (2017, background): “For example, different initial weights can be used for individual neural networks [28], whereas different split selections can be applied to individual decision trees [37].”
Learning and Evolution (1999, background): “…in the case of our robots we should consider two facts: a) initial conditions (e.g., initial weights) can determine the course of learning by error minimization (Kolen and Pollack, 1990), and b) evolution can select appropriate initial weights for learning (Belew, McInerney and Schraudolph, 1991).”
Learning and Evolution in Neural Networks (1994, background): “As Kolen & Pollack (1990) have demonstrated, backpropagation learning is sensitive to initial conditions (initial weight matrices).”
Round Robin Classification (2002, methodology): “Other techniques include the use of diﬀerent feature subsets (Bay, 1999), to exploit the randomness of the base algorithms (Kolen and Pollack, 1991), possibly by artiﬁcially randomizing their behavior (Dietterich, 2000b), or to use multiple representations of the domain objects, for example by…”
The use of artificial neural networks to analyze and predict alongshore sediment transport (2010, background): “algorithms, especially for the backpropagation case (Kolen and Pollack, 1990), can be extremely sensitive to the initial values assigned to biases and weights.”