Gradient-based learning applied to document recognition

By Yann LeCun, L. Eon Bottou, Yoshua Bengio, Patrick Haffner. 1998

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Adversarial Nets (2014, methodology): “The reported numbers on MNIST are the mean log-likelihood of samples on test set, with the standard error of the mean computed across examples.”
Structured Attention Networks (2017, background): “…at the ﬁnal, output layer has been shown to be important in many deep learning applications, most notably in seminal work on graph transformers (LeCun et al., 1998), key work on NLP (Collobert et al., 2011), and in many other areas (Peng et al., 2009; Do & Arti´eres, 2010; Jaderberg et al.,…”
Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation (2016, methodology): “The deep topology brings difficulties for the model training, especially when first order methods such as stochastic gradient descent (SGD) (LeCun et al., 1998) are used.”
Sequence to Sequence Learning with Neural Networks (2014, background): “Deep Neural Networks (DNNs) are extremely powerful machine learning models that achieve excellent performance on difficult problems such as speech recognition [13, 7] and visual object recognition [19, 6, 21, 20].”
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015, methodology): “As shown in (LeCun et al., 1998b), such normalization speeds up convergence, even when the features are not decorrelated.”
FitNets: Hints for Thin Deep Nets (2014, methodology): “As a sanity check for the training procedure, we evaluated the proposed method on the MNIST dataset (LeCun et al., 1998).”
Deeply-Supervised Nets (2014, methodology): “hows test error w.r.t. the training sample size. (c) training and testing error comparison. We ﬁrst validate the effectiveness of the proposed DSN on the MNIST handwritten digits classiﬁ- cation task [17], a widely and extensively adopted benchmark in machine learning. MNIST dataset consists of imag”
Network In Network (2013, background): “t, we obtain a test error rate of 2.35%, which sets the new state-of-the-art. We compare our result with methods that did not augment the data, the comparison is shown in Table 3. 4.5 MNIST The MNIST [1] dataset consists of hand written digits 0-9 which are 28x28 in size. There are 60,000 training i”
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, methodology): “The features used in the UVA detection system [39], for example, are two orders of magnitude larger than ours (360k vs. 4k-dimensional).”
Improving neural networks by preventing co-adaptation of feature detectors (2012, background): “Performance on the test set can be greatly improved by enhancing the training data with transformed images [5] or by wiring knowledge about spatial transformations into a convolutional neural network [17] or by using generative pre-training to extract useful features from the training images without”
Deep AutoRegressive Networks (2013, background): “Local connectivity can be either fully convolutional (LeCun et al., 1998) or use less weight sharing.”
Pylearn2: a machine learning research library (2013, methodology): “…the dataset used for the NIPS 2011 workshops challenge in representation learning [32], the CIFAR-10 and CIFAR-100 datasets [29], the MNIST dataset [34], some of the MNIST variations datasets [31], the NORB dataset [35], the Street View House Numbers dataset [36], the Toronto Faces Database [50],…”
Better Mixing via Deep Representations (2012, methodology): “The experiments have been performed on the MNIST digits dataset (LeCun et al., 1998) and the Toronto Face Database (Susskind et al.”
Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “Whereas it took around 15 years to achieve “near-human performance” on MNIST (LeCun et al., 1998; Cires¸an et al., 2012; Wan et al., 2013) and approximately 7 years to surpass humans on ImageNet (Deng et al., 2009; Russakovsky et al., 2015; He et al., 2016), the GLUE benchmark did not last as long a”
A Convolutional Neural Network for Modelling Sentences (2014, methodology): “The max pooling operator is a non-linear subsampling function that returns the maximum of a set of values (LeCun et al., 1998).”
Character-level Convolutional Networks for Text Classification (2015, background): “On the other hand, many researchers have found convolutional networks (ConvNets) [17] [18] are useful in extracting information from raw signals, ranging from computer vision applications to speech recognition and others.”