Large Automatic Learning, Rule Extraction, and Generalization

By John S. Denker, D. B. Schwartz, Ben S. Wittner, Sara A. Solla, Richard E. Howard, L. Jackel, John J. Hopfield. 1987

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Generalization and network design strategies (1989, methodology): “…reducmg the s1ze of the space of poss1ble functwns that the network can generate, without overly reducmg 1ts computatwnal power Theoretical studies (Denker et al , 1987) (Patarnello and Carnevah, 1987) have shown that the likehhood of correct generahzat10n depends on the size of the hypothesis…”
Action and Perception as Divergence Minimization (2020, methodology): “Figure 3 shows variational inference for the example of supervised learning using a BNN (Denker et al., 1987; MacKay, 1992a; Blundell et al., 2015).”
Backpropagation Applied to Handwritten Zip Code Recognition (1989, background): “…in shift-invariance, but vastly reduces the entropy, the Vapnik-Chervonenkis dimensionality, and the number of free parameters, thereby proportionately reducing the amount of training data required to achieve a given level of generalization performance (Denker et al. 1987; Baum and Haussler 1989).”
A survey of uncertainty in deep neural networks (2021, background): “Bayesian Neural Networks (BNNs) [106], [107], [108] have”
Bayesian Regularization and Pruning Using a Laplace Prior (1995, methodology): “This paper uses the method of integrating over hyperparameters ( Buntine and Weigend 1991 ).”
Optimal Brain Damage (1989, background): “It is known from theory (Denker et al., 1987; Baum and Haussler, 1989; Solla et al., 1990) and experience (Le Cun, 1989) that, for a fixed amount of training data, networks with too many weights do not generalize well.”
Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference (2001, background): “A number of people have considered the extraction of symbolic knowledge from trained neural networks for both feedforward and recurrent neural networks [12, 44, 57, 44].”
Uncertainty in Deep Learning (2016, background): “In essence, [Denker et al., 1987, page 904] proposed placing a prior distribution over the space of weights (and used a uniform distribution on a compact space).”