Backpropagation Applied to Handwritten Zip Code Recognition

By Yann LeCun, Bernhard E. Boser, John S. Denker, D. Henderson, Richard E. Howard, Wayne E. Hubbard, L. Jackel. 1989

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Residual Learning for Image Recognition (2015, methodology): “15 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. 1. Introduction Deep convolutional neural networks [22,21] have led to a series of breakthroughs for image classiﬁcation [21, 50,40]. Deep networks nat”
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015, background): “The RPN, which is naturally implemented as a fully-convolutional network [14], can be trained end-to-end by back-propagation and stochastic gradient descent (SGD) [12].”
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, background): “The last two years, huge improvements have been observed on this task thanks to the recent advances of deep learning community [19, 2, 15].”
Fast R-CNN (2015, methodology): “The use of deep ConvNets [15, 17] for image classiﬁ-cation [13] and object detection [9, 18] has led to signiﬁ-cant gains in accuracy.”
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, methodology): “Convolutional neural networks (CNNs) [19, 18] have demonstrated recognition accuracy better than or comparable to humans in several visual recognition tasks, including recognizing traffic signs [3], faces [34, 32], and handwritten digits [3, 36].”
Convolutional neural networks at constrained time cost (2014, background): “Convolutional neural networks (CNNs) [15, 14] have recently brought in revolutions to the computer vision area.”
Fully convolutional networks for semantic segmentation (2014, methodology): “lation layers and a spatial loss (as in Figure1) produces an efﬁcient machine for end-to-end pixelwise learning. 3.1 Adapting classiﬁers for dense prediction Typical recognition nets, including LeNet [21], AlexNet [1], and its deeper successors [2], [3], ostensibly take ﬁxedsized inputs and produce ”
Deeply-Supervised Nets (2014, methodology): “The goal of deep neural networks, specifically convolutional neural networks (CNN) [16], is to learn layers of filters and weights for minimization of output layer classification error.”
Going deeper with convolutions (2014, methodology): “We propose a deep convolutional neural network architecture codenamed Inception, which was responsible for setting the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).”
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014, methodology): “Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training the detectors.”
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, methodology): “Second, the feature vectors computed by the CNN are low-dimensional when compared to other common approaches, such as spatial pyramids with bag-of-visual-word encodings.”
A Generative Process for Contractive Auto-Encoders (2012, background): “This behavior is to be contrasted with a very successful element of many deep architectures (especially for machine vision tasks), which is the addition of pooling layers in convolutional networks (LeCun et al., 1989).”
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020, methodology): “There has also been a lot of interest in combining convolutional neural networks (CNNs) with forms of self-attention, e.g. by augmenting feature maps for image classiﬁcation (Bello et al., 2019) or by further processing the output of a CNN using self-attention, e.g. for object detection (Hu et al., ”
Masked Autoencoders Are Scalable Vision Learners (2021, background): “In vision, convolutional networks [29] were dominant over the last decade [28].”
U-Net: Convolutional Networks for Biomedical Image Segmentation (2015, background): “While convolutional networks have already existed for a long time [8], their success was limited due to the size of the available training sets and the size of the considered networks.”
Mask R-CNN (2017, methodology): “Related Work R-CNN: The Region-based CNN (R-CNN) approach [13] to bounding-box object detection is to attend to a manageable number of candidate object regions [38, 20] and evaluate convolutional networks [25, 24] independently on each RoI.”
Momentum Contrast for Unsupervised Visual Representation Learning (2019, background): “The encoder can be any convolutional neural network [39].”
Rethinking ImageNet Pre-Training (2018, background): “Deep convolutional neural networks [21, 23] revolutionized computer vision arguably due to the discovery that feature representations learned on a pre-training task can transfer useful information to target tasks [9, 6, 50].”
What makes ImageNet good for transfer learning? (2016, background): “It has become increasingly common within the computer vision community to treat image classification on ImageNet [37] not as an end in itself, but rather as a “pretext task” for training deep convolution neural networks (CNNs [27, 25]) to learn good general-purpose features.”