Best practices for convolutional neural networks applied to visual document analysis

By Patrice Y. Simard, David Steinkraus, John C. Platt. 2003

3,058 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Model Architectures, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

In Signal Processing

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Data augmentation is a widely used method for generating additional data to improve machine learning systems, for image classiﬁcation [43, 23, 7, 54], object detection [13], instance segmentation [10], and speech recognition [21, 16, 36].”
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (2016, methodology): “Multi-layer perceptron [22] vector 1.60 LeNet5 [12] image 0.80 Ours PointNet point set 0.78 We train a modiﬁed version of our segmentation Point-Net in a supervised manner to regress to the ground-truth point normals.”
MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, background): “For example, in image classification, it is common to elastically deform or add noise to an input image, which can dramatically change the pixel content of an image without altering its label [7, 43, 10].”
Representation Learning: A Review and New Perspectives (2012, background): “Using both affine and elastic deformations (Simard et al., 2003), with plain old stochastic gradient descent, they reach a record 0.32% classification error rate.”
AutoAugment: Learning Augmentation Policies from Data (2018, background): “For example, on MNIST, most top-ranked models use elastic distortions, scale, translation, and rotation [2, 30–32].”
Multi-column deep neural networks for image classification (2012, methodology): “1a) have 2-dimensional layers of winner-take-all neurons with overlapping receptive fields whose weights are shared [19, 1, 32, 7].”
Grid Long Short-Term Memory (2015, methodology): “tion. We 12 Test Error (%) Wan et al. [38] 0.28 Graham [11] 0.31 Untied 3-LSTM 0.32 Ciresan et al. [5] 0.35 Untied 3-LSTM with ReLU (*) 0.36 Mairar et al. [29] 0.39 Lee et al. [27] 0.39 Simard et al. [32] 0.4 Graham [10] 0.44 Goodfellow et al. [9] 0.45 Visin et al. [37] 0.45 Lin et al. [28] 0.47 Fig”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, background): “Classical data augmentation techniques (such as mirroring, cropping) have been widely used in past few years [22, 35] to provide high capacity models sufficient data to learn from.”
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition (2010, methodology): “We combine afﬁne (rotation, scaling and horizontal shearing) and elastic deformations, characterized by the following real-valued parameters: • σ and α : for elastic distortions emulating uncontrolled oscillations of hand muscles (for details see Simard et al., 2003);”
Compressing Neural Networks with the Hashing Trick (2015, methodology): “Arguably the most popular method is the widely used convolutional neural network (Simard et al., 2003).”
Toward Causal Representation Learning (2021, neutral): “The second approach, often coupled with the previous one, is to rely on data augmentation to increase the diversity of the data by “augmenting” it through a certain type of artificially generated interventions [10], [140], [234].”
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (2017, methodology): “Multi-layer perceptron [24] 1.60 LeNet5 [11] 0.80 Network in Network [13] 0.47 PointNet (vanilla) [20] 1.30 PointNet [20] 0.78 Ours 0.51 from PointNet (vanilla) and PointNet to our method.”
Machine Learning Models that Remember Too Much (2017, background): “Data augmentation is a very common practice for boosting the performance of machine learning models [41, 69].”
Deep learning in neural networks: An overview (2014, methodology): “Its units have hand-crafted weights; biologically plausible learning ruleswere later proposed for similar models (e.g., Serre, Riesenhuber, Louie, & Poggio, 2002; Teichmann, Wiltschut, & Hamker, 2012).”
UniformAugment: A Search-free Probabilistic Data Augmentation Approach (2020, methodology): “For example random ﬂip, and color distortions are commonly used for natural image datasets such as CIFAR-10 [10] and ImageNet [5], while elastic distortions and scalings are more common on MNIST [14,15] dataset.”