Between-Class Learning for Image Classification

By Yuji Tokozume, Yoshitaka Ushiku, Tatsuya Harada. 2017

222 citations recorded by the source corpus.

In this paper, we propose a novel learning method for image classification called Between-Class learning (BC learning)1. We generate between-class images by mixing two images belonging to different classes with a random ratio. We then input the mixed image to the model and train the model to output the mixing ratio. BC learning has the ability to impose constraints on the shape of the feature distributions, and thus the generalization ability is improved. BC learning is originally a method developed for sounds, which can be digitally mixed. Mixing two image data does not appear to make sense; however, we argue that because convolutional neural networks have an aspect of treating input data as waveforms, what works on sounds must also work on images. First, we propose a simple mixing method using internal divisions, which surprisingly proves to significantly improve performance. Second, we propose a mixing method that treats the images as waveforms, which leads to a further improvement in performance. As a result, we achieved 19.4% and 2.26% top-1 errors on ImageNet-1K and CIFAR-10, respectively.2

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “Mixup: CutMix shares similarity with Mixup [47, 40] in that both combines two samples, where the ground truth label of the new sample is given by the linear interpolation of one-hot labels.”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, background): “Rather than implanting one portion of an image inside another, Mixup produces an elementwise convex combination of two images (Zhang et al., 2017; Tokozume et al., 2018).”
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation (2019, background): “Another strategy is to mix multiple images of different classes either by convex combinations [36,30] or by creating a patchwork from them [34].”
Style transfer-based image synthesis as an efficient regularization technique in deep learning (2019, methodology): “GANs are found to be very useful in many different image generation and manipulation problems like text-to-image synthesis [22], super-resolution (generating high-resolution image out of low-resolution one) [23], image-to-image translation (e.g. convert sketches to images) [24], image blending (mixi”
Shakedrop Regularization for Deep Residual Learning (2018, background): “Mixup [38], BC learning [30], and RICAP [29] generate new data and their corresponding class labels by interpolating two or more data.”
Learning Placeholders for Open-Set Recognition (2021, methodology): “In this section, we give the detailed pseudo code for manifold mixup for data placeholders and the performance comparison between manifold mixup and vanilla mixup [39, 30].”
Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data (2019, background): “Mixup [46, 38] combined two samples linearly in pixel level, where the target of the synthetic image was the linear combination of one-hot label encodings.”
Manifold Mixup: Better Representations by Interpolating Hidden States (2018, methodology): “Manifold Mixup is a generalization of Input Mixup, the idea of building random interpolations between training examples and perform the same interpolation for their labels (Zhang et al., 2018; Tokozume et al., 2018).”
SpeechMix - Augmenting Deep Sound Recognition Using Hidden Space Interpolations (2020, background): “Mixup has been demonstrated to work well on image data [10, 11, 12, 13] and text classification [14, 15].”
MixUp as Locally Linear Out-Of-Manifold Regularization (2018, methodology): “Our work closely relates to approaches linearly interpolating examples and labels (Zhang et al. 2017; DeVries and W. Taylor 2017; Verma et al. 2018; Tokozume, Ushiku, and Harada 2017).”
Towards Domain-Agnostic Contrastive Learning (2020, background): “Mixup(Zhang et al., 2018; Tokozume et al., 2017) and its numerous variants(Verma et al., 2019a; Yun et al., 2019; Faramarzi et al., 2020) have seen remarkable success in supervised learning problems, as well other problems such as semi-supervised learning (Verma et al., 2019b; Berthelot et al.,…”
Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis (2020, methodology): “Data augmentation is a widely used technique for generating additional data to train machine learning systems [50, 48, 27, 57, 11, 58].”
FMix: Enhancing Mixed Sample Data Augmentation (2020, methodology): “Recently, a plethora of approaches to Mixed Sample Data Augmentation (MSDA) have been proposed which obtain state-of-the-art results, particularly in classification tasks [3, 53, 42, 43, 20, 51, 40, 39].”
Improved Mixed-Example Data Augmentation (2018, methodology): “Furthermore, previous work [27, 24] has already shown strong correlations in improvements across model architectures.”
Does Knowledge Distillation Really Work? (2021, background): “Mixup Mixup is an effective regularization technique originally proposed to increase generalization and robustness of deep networks [49, 43].”
Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer (2018, background): “The first is the idea of linearly interpolating between different randomly drawn examples and similarly interpolating the labels (Zhang et al., 2017; Tokozume et al., 2017).”
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning (2022, methodology): “BC Learning [29] 2018 Combine two images and their labels.”
Augmenting NLP models using Latent Feature Interpolations (2020, methodology): “Mixup (Zhang et al., 2017; Verma et al., 2018; Tokozume et al., 2018; Yun et al., 2019) has shown improvements in the accuracy of image classification models in Computer Vision.”