Learning representations by back-propagation errors, nature

By D. Rumelhart, Geoffrey E. Hinton, R. William. 1986

Subfields: Model Architectures, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Optimization & Training

Citations in the corpus

Semi-supervised Sequence Learning (2015, background): “Recurrent neural networks (RNNs) are powerful tools for mod eling sequential data, yet training them by back-propagation through time [36, 26] can be difficu lt [8].”
Distributed Representations of Sentences and Documents (2014, background): “Distributed representations for words were ﬁrst proposed in (Rumelhart et al., 1986) and have become a successful paradigm, especially for statistical language modeling (El-man, 1990; Bengio et al., 2006; Mikolov, 2012).”
Machine learning of molecular electronic properties in chemical compound space (2013, background): “Errors are back-propagated [63] from the top layer back to the inputs in order to update all parameters of the model.”
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units (2015, methodology): “However, training RNNs by using back-propagation through t ime [30] to compute error-derivatives can be difficult.”
Breaking the Curse of Dimensionality with Convex Neural Networks (2014, background): “(d) Single hidden-layer neural networks: f(x) = ∑k j=1 σ(w ⊤ j x + bj), where k is the number of units in the hidden layer (see, e.g., Rumelhart et al., 1986; Haykin, 1994).”
Deep Learning of Representations: Looking Forward (2013, background): “Before 2006, it was believed that training deep supervised neural networks (Rumelhart et al., 1986) was too difficult (and indeed did not work).”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, methodology): “Common training methods include back-propagation [Rumelhart et al., 1986], which is a form of gradient descend method to use the loss to update weights in different layers.”
How Learning Can Guide Evolution (1996, methodology): “More sophisticated learning procedures could be used in these cases (e.g. Rumelhart, Hinton, and Williams, 1986).”
A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition (2011, unknown): “These previous comparisons have been performed with diﬀerent datasets in diﬀerent conﬁgurations us-ing diﬀerent forecasting methods, such as Multiple Linear Regression, Artiﬁcial Neural Networks, Hidden Markov Models and Nearest Neighbors.”
Bayesian Methods for Adaptive Models (2011, background): “There are many knobs on the black box of ‘backprop’ (learning by back–propagation of errors [66]).”
Synaptic modification by correlated activity: Hebb's postulate revisited. (2001, background): “…synapse specificity has been a “central dogma” in the concept of synaptic modification, a popular network model of supervised learning adopted a non-local rule, with which signals for synaptic modification can propagate backward through connections in multilayer networks (Rumelhart et al 1986a,b).”
Maturity changes in Norwegian spring-spawning herring before, during, and after a major population collapse (2004, methodology): “These were discriminant analysis (DA), a conventional statistical procedure; and the relatively new methodology of artificial neural networks (NN), trained by back-propagation (Rumelhart et al., 1986).”
Transmembrane helices predicted at 95% accuracy (1995, methodology): “Training was performed with the usual gradient descent (also known as back-propagation Rumelhart et al., 1986)): ∆ ∆J Jij ij (t +1) = E (t) J (t) + (t) ij ε α⋅ ⋅ (1) where t is the algorithmic time step (i.e. change of all connections for one pattern), E is the error, given by the difference…”