Handwritten Digit Recognition with a Back-Propagation Network

By Yann LeCun, Bernhard E. Boser, John S. Denker, D. Henderson, Richard E. Howard, Wayne E. Hubbard, L. Jackel. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Exploring the Limits of Language Modeling (2016, methodology): “For inference, the vectors e w can be precomputed, so there is no computational complexity increase w.r.t. the regular Soft-max.”
A Decomposable Attention Model for Natural Language Inference (2016, background): “The dominating trend in these models is to build complex, deep text representation models, for example, with convolutional networks (LeCun et al., 1990, CNNs henceforth) or long short-term memory networks (Hochreiter and Schmidhuber, 1997, LSTMs henceforth) with the goal of deeper sentence…”
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks (2013, background): “…image is a tasks to which Convolutional Networks (ConvNets) [17] have been applied for many years, whether the objects were handwritten characters [16], house numbers [24], textureless toys [18], trafﬁc signs [3, 26], objects from the Caltech-101 dataset [14], or objects from the 1000-category…”
Making Convolutional Networks Shift-Invariant Again (2019, background): “Early networks did employ a form of blurred-downsampling – average pooling (LeCun et al., 1990).”
HyperNetworks (2016, neutral): “On image classiﬁcation with CIFAR-10, hypernetworks, when being used to generate weights for a deep con-vnet (LeCun et al., 1990), obtain respectable results compared to state-of-the-art models while having fewer learnable parameters.”
End-to-End Training of Deep Visuomotor Policies (2015, background): “rol layers can be adapted to imperfect perception. We represent our policies with convolutional neural networks (CNNs). CNNs have a long history in computer vision and deep learning (Fukushima, 1980; LeCun et al., 1989; Schmidhuber, 2015), and have recently gained prominence due to excellent results”
The Reversible Residual Network: Backpropagation Without Storing Activations (2017, background): “Over the last five years, deep convolutional neural networks have enabled rapid performance improvements across a wide range of visual processing tasks [19, 26, 20].”
Scalable agent alignment via reward modeling: a research direction (2018, background): “Yet further inductive biases are necessary to solve many tasks; for instance, convolutional neural networks (LeCun et al., 1990) vastly outperform multilayer perceptrons in computer vision applications because of their spatial invariance.”
Unified Contrastive Learning in Image-Text-Label Space (2022, methodology): “With this goal, numerous works have pushed the image recognition performance from different directions, such as data scale from MNIST [36] to ImageNet1K [10], model architectures from convolutional neural networks (CNNs) [23, 26, 34, 35, 40, 52, 53] to Transformers [15, 43, 57, 61, 64, 67, 72], and ”
Deep Learning Face Attributes in the Wild (2014, methodology): “We adopt softmax for object classification, which is optimized by stochastic gradient descent (SGD) with back-propagation (BP) [16].”
Learning to See by Moving (2015, methodology): “The visual system of the agent is modelled by a randomly initialized Convolutional Neural Network (CNN, [25]).”
Character-Aware Neural Language Models (2015, background): “CNNs (LeCun et al. 1989) have achieved state-of-the-art results on computer vision (Krizhevsky, Sutskever, and Hinton 2012) and have also been shown to be effective for various NLP tasks (Collobert et al. 2011).”
Controllable Abstractive Summarization (2017, methodology): “The encoder and decoder are based on convolutional networks (LeCun et al., 1990).”
Unsupervised Learning by Predicting Noise (2017, background): “In recent years, convolutional neural networks, or con-vnets (Fukushima, 1980; LeCun et al., 1989) have pushed the limits of computer vision (Krizhevsky et al., 2012; He et al., 2016), leading to important progress in a variety of tasks, like object detection (Girshick, 2015) or image segmentation…”
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning (2016, background): “Convolutional neural networks (ConvNets) [1, 2] achieve state-of-the-art accuracy on a variety of computer vision tasks, including classification, object localization, detection, recognition and scene labeling [3, 4].”
Optimization Methods for Large-Scale Machine Learning (2016, background): “At the same time, convolutional neural networks (CNNs) have proved to be very effective for computer vision and signal processing tasks [85, 22, 86, 82].”
Unsupervised Learning of Visual Representations Using Videos (2015, methodology): “ed to semantic/3D/functional categories. However, the last three years have seen the resurgence of learning visual representations directly from pixels themselves using the deep learning and ConvNets [25,21,20]. At the heart of ConvNets is a completely supervised learning paradigm. Often millions of”
Compressing Deep Convolutional Networks using Vector Quantization (2014, background): “Deep convolutional neural networks (Krizhevsky et al., 2012; LeCun et al., 1990; Szegedy et al., 2014; Simonyan & Zisserman, 2014) has recently achieved signiﬁcant progress and have become the gold standard for object recognition, image classiﬁcation, and retrieval.”
Designing deep networks for surface normal estimation (2014, methodology): “In this paper, we use CNNs for the task of predict- ing surface normals from a single image.”
Deep learning in neural networks: An overview (2014, background): “Some of the GPUbased FNN implementations (Sections 5.16–5.19) have greatly contributed to recent successes in contests for pattern recognition (Sections 5.19–5.22), image segmentation (Section 5.21), and object detection (Sections 5.21–5.22).”