APAC: Augmented PAttern Classification with Neural Networks

By Ikuro Sato, Hiroki Nishimura, Kensuke Yokoi. 2015

143 citations recorded by the source corpus.

Deep neural networks have been exhibiting splendid accuracies in many of visual pattern classification problems. Many of the state-of-the-art methods employ a technique known as data augmentation at the training stage. This paper addresses an issue of decision rule for classifiers trained with augmented data. Our method is named as APAC: the Augmented PAttern Classification, which is a way of classification using the optimal decision rule for augmented data learning. Discussion of methods of data augmentation is not our primary focus. We show clear evidences that APAC gives far better generalization performance than the traditional way of class prediction in several experiments. Our convolutional neural network model with APAC achieved a state-of-the-art accuracy on the MNIST dataset among non-ensemble classifiers. Even our multilayer perceptron model beats some of the convolutional models with recently invented stochastic regularization techniques on the CIFAR-10 dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Stochastic Optimization of Plain Convolutional Neural Networks with Simple Methods (2020, methodology): “d Dropout= 0.8. Table 3. Top MNIST Results. Method Error rate DropConnectWan et al. (2013) [7] 0.21% Multi-column DNN for Image Classification Ciregan et al. (2012) [18] 0.23% APAC Sato et al. (2015) [17] 0.23% Generalizing Pooling Functions in CNN Lee et al. (2016) [15] 0.29% Stochastic Optimizatio”
Randaugment: Practical automated data augmentation with a reduced search space (2019, background): “On MNIST, elastic distortions across scale, position, and orientation have been applied to achieve impressive results [43, 4, 49, 42].”
FILIP: Fine-grained Interactive Language-Image Pre-Training (2021, methodology): “We apply AutoAugment (Krizhevsky et al., 2012; Sato et al., 2015; Cubuk et al., 2019; Hoffer et al., 2020) for image augmentation, following the SOTA vision recognition methods (Touvron et al., 2021; Xie et al., 2020b).”
AutoAugment: Learning Augmentation Policies from Data (2018, background): “For example, on MNIST, most top-ranked models use elastic distortions, scale, translation, and rotation [2, 30–32].”
UniformAugment: A Search-free Probabilistic Data Augmentation Approach (2020, methodology): “For example random ﬂip, and color distortions are commonly used for natural image datasets such as CIFAR-10 [10] and ImageNet [5], while elastic distortions and scalings are more common on MNIST [14,15] dataset.”
Automatically Learning Data Augmentation Policies for Dialogue Tasks (2019, neutral): “…work that employs data augmentation in both computer vision (Simard et al., 2003; Krizhevsky et al., 2012; Cires¸an et al., 2012; Wan et al., 2013; Sato et al., 2015; DeVries and Taylor, 2017; Tran et al., 2017; Lemley et al., 2017) and NLP (F¨urstenau and Lapata, 2009; Sen-nrich et al., 2016;…”
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules (2019, methodology): “…as random cropping, ﬂipping, rotating, scaling, and translating are used by top performing models for training on MINST, CIFAR-10, and ImageNet datasets (Simard et al., 2003; Ciresan et al., 2012; Wan et al., 2013; Sato et al., 2015; Krizhevsky et al., 2012; Lin et al., 2013; Han et al., 2016).”
Fast AutoAugment (2019, methodology): “On the benchmark image dataset, such as CIFAR and ImageNet, random crop, flip, rotation, scaling, and color transformation, have been performed as baseline augmentation methods [10, 21, 30].”
Understanding Data Augmentation for Classification: When to Warp? (2016, background): “The issue of the difference between real and synthetic examples is discussed by [14], in using a classifier that was trained with augmented data only.”
Generalization in Deep Learning (2017, methodology): “23% for MNIST with a single model (Sato et al., 2015) (and 0.”
Learning Data Augmentation Strategies for Object Detection (2019, background): “For example, state-of-the-art models trained on MNIST use elastic distortions which effect scale, translation, and rotation [42, 4, 47, 40].”
Simple convolutional neural network on image classification (2017, methodology): “APAC: Augmented Pattern Classification with Neural Networks[22] 0.”
AutoAugment: Learning Augmentation Strategies From Data (2019, background): “For example, on MNIST, most top-ranked models use elastic distortions, scale, translation, and rotation [54, 8, 62, 52].”
Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures (2016, methodology): “Inspired by Hinton et al. (2015), Romero et al. (2014) proposed a novel architecture to address what they referred to as not taking advantage of depth in the previous works related to Convolutional Neural Networks model compression.”
Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data (2019, background): “When dealing with images, it often includes the application of rotation, translation, blurring, and other modifications [4, 51, 42] to existing images that allow a network to better generalize [43].”
Tradeoffs in Data Augmentation: An Empirical Study (2021, methodology): “Indeed, elastic distortions as well as distortions in the scale, position, and orientation of training images have been used on MNIST (Ciregan et al., 2012; Sato et al., 2015; Simard et al., 2003; Wan et al., 2013), while horizontal flips, random crops, and random distortions to color channels have ”
Affinity and Diversity: Quantifying Mechanisms of Data Augmentation (2020, methodology): “Indeed, elastic distortions as well as distortions in the scale, position, and orientation of training images have been used on MNIST (Ciregan et al., 2012; Sato et al., 2015; Simard et al., 2003; Wan et al., 2013), while horizontal flips, random crops, and random distortions to color channels have ”
Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data (2019, methodology): “Basic image augmentation methods mainly include elastic distortions, scale, translation, and rotation [7, 35].”