Connectionist Learning Procedures

By Geoffrey E. Hinton. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

A Convolutional Neural Network for Modelling Sentences (2014, methodology): “Then we describe the operation of onedimensional convolution and the classical TimeDelay Neural Network (TDNN) (Hinton, 1989; Waibel et al., 1990).”
Regularizing and Optimizing LSTM Language Models (2017, background): “TAR falls under the broad category of slowness regularizers (Hinton, 1989; Földiák, 1991; Luciw & Schmidhuber, 2012; Jonschkowski & Brock, 2015) which penalize the model from producing large changes in the hidden state.”
Deep learning in neural networks: An overview (2014, methodology): “Compare an early survey (Hinton, 1989) and the somewhat related Recursive Auto-Associative Memory (RAAM) (Melnik, Levy, & Pollack, 2000; Pollack, 1988, 1990), originally used to encode sequential linguistic structures of arbitrary size through a fixed number of hidden units.”
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018, methodology): “Convolutional networks (LeCun et al., 1989) have been applied to sequences for decades (Sejnowski & Rosenberg, 1987; Hinton, 1989).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “In the case where the inputs are not images or easily visualizable, or to get a sense of the weight values in diﬀerent hidden units, Hinton diagrams (Hinton, 1989) are also very useful, using small squares whose color (black or white) indicates a weight’s sign and whose area represents its…”
Deep Learning using Linear Support Vector Machines (2013, background): “The primal L2-SVM objective was proposed 3 years before the invention of SVMs (Hinton, 1989)!”
Deep Directed Generative Autoencoders (2014, background): “These representations are distributed (Hinton, 1989), meaning that at each level there are many variables or features, which together can take a very large number of conﬁgurations.”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “To elaborate on the need for an eﬃcient means of gradient computation in the brain, we will ﬁrst place backpropagation into its computational context (Hinton, 1989; Baldi and Sadowski, 2015).”
Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm (1996, background): “The origin of the phase-based activations that are central to the GeneRec algorithm touches at the heart of perhaps the most controversial aspect of error-driven learning in the cortex: “where does the teaching signal come from?” In GeneRec, the teaching signal is just the plus-phase activation stat”
What Size Net Gives Valid Generalization? (1989, background): “E of future test examples drawn from the same distribution.”
Discrete-State Variational Autoencoders for Joint Discovery and Factorization of Relations (2016, methodology): “In order to implement the desiderata sketched in the previous section, we take an inspiration from a framework popular in the neural network community, namely autoencoders (Hinton, 1989).”
Iterative weighted least squares algorithms for neural networks classifiers (1992, methodology): “It is experimentally shown that the algorithms converge with fewer iterations than usual BP algorithm.”
Unsupervised Learning of Visual Representations using Videos (2015, methodology): “Zemel and Hinton (1991) used mutual information to discover viewpoint invariant representations of an object.”
Revisiting Activation Regularization for Language RNNs (2017, background): “This broad concept falls under the broad concept of slowness regularization (Hinton, 1989; Földiák, 1991; Luciw & Schmidhuber, 2012; Jonschkowski & Brock, 2015; Wen et al., 2015) which attempts to minimize L(f(xt), f(xt+1)) where L is a loss function describing the distance between f(xt) and…”
Age of acquisition effects in adult lexical processing reflect loss of plasticity in maturing systems: insights from connectionist networks. (2000, background): “One consequence of this is that the capacity of the network to generalize to new inputs is improved (Hinton, 1989). Thus, Plaut et al. (1996) introduced weight decay into some of their simulations of reading aloud to improve the ability of a network trained to associate written with spoken word form”
Convolutional Sequence Modeling Revisited (2018, methodology): “D convolutional filter over the data (and stack such layers together) to predict future elements of a sequence from past ones (Hinton, 1989; LeCun et al., 1995).”
Combining Symbolic and Neural Learning (1994, background): “Rather than minimizing mean-squared error, the cross-entropy error function (Hinton, 1989) is a better choice for knowledge-based networks (see Towell (1992) for an explanation).”