CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
By Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, Y. Yoo. 2019
6,079 citations recorded by the source corpus.
Regional dropout strategies have been proposed to enhance performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to attend on less discriminative parts of objects (e.g. leg as opposed to head of a person), thereby letting the network generalize better and have better object localization capabilities. On the other hand, current methods for regional dropout removes informative pixels on training images by overlaying a patch of either black pixels or random noise. Such removal is not desirable because it suffers from information loss causing inefficiency in training. We therefore propose the CutMix augmentation strategy: patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches. By making efficient use of training pixels and retaining the regularization effect of regional dropout, CutMix consistently outperforms state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on ImageNet weakly-supervised localization task. Moreover, unlike previous augmentation methods, our CutMix-trained ImageNet classifier, when used as a pretrained model, results in consistent performance gain in Pascal detection and MS-COCO image captioning benchmarks. We also show that CutMix can improve the model robustness against input corruptions and its out-of distribution detection performance.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Jianfeng Gao
- Mohit Bansal
- Yoshua Bengio
- Dacheng Tao
- Percy Liang
- Sergey Levine
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Alan L. Yuille
In Optimization & Training
- Quoc V. Le
- Yoshua Bengio
- Soham De
- Samuel L. Smith
- Andrew Brock
- Alan L. Yuille
- Frank Hutter
- Mingxing Tan
Citations in the corpus
- Masked Autoencoders Are Scalable Vision Learners (2021, methodology): “[52] 0.1 mixup [69] 0.8 cutmix [68] 1.0 drop path [30] 0.1 (B/L) 0.2 (H) Partial fine-tuning.”
- Big Transfer (BiT): General Visual Representation Learning (2019, background): “Numerous improvements to deep network training have recently been introduced, e.g. [55,62,26,35,22,1,64,67,54,60].”
- Fixing the train-test resolution discrepancy (2019, neutral): “4 ResNet-50 CutMix [44] 224 224 25.”
- Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, methodology): “resnet101_cutmix [108].”
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “The space of augmentations that help robustness includes various types of noise [22, 27, 21], highly unnatural image transformations [10, 38, 39], or compositions of simple image transformations such as Python Imaging Library operations [4, 15].”
- Natural Adversarial Examples (2019, background): “CutMix [55] superimposes images regions within other images and yields 7.”
- VOLO: Vision Outlooker for Visual Recognition (2021, methodology): “We do not use MixUp [72] or CutMix [70] as they conflict with MixToken.”
- Training data-efficient image transformers & distillation through attention (2020, background): “Regularization like Mixup [57] and Cutmix [56] improve performance.”
- Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, methodology): “For ImageNet experiments, we use CutMix (Yun et al., 2019) and MixUp (Zhang et al.”
- A U-Net Based Discriminator for Generative Adversarial Networks (2020, methodology): “In contrast to [47], the class label c ∈ {0, 1} for the new CutMix image x̃ is set to be fake, i.”
- A Survey of Data Augmentation Approaches for NLP (2021, methodology): “Zhang et al. (2019b) explore multi-task transfer, or the use of annotated data from other tasks.”
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “, 2017), or CutMix (Yun et al., 2019) of varying strengths and random cropping with a minimum crop size of 5%, 70%, 90%, or 100% of the full image.”
- High-Performance Large-Scale Image Recognition Without Normalization (2021, methodology): “The combination of MixUp, CutMix, and RA results in an intense level of augmentation which progressively benefits NFNets, but does not appear to benefit other models like EfficientNets over a baseline of just using well-tuned RA.”
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, methodology): “We include most of the augmentation and regularization strategies of [63] in training, including RandAugment [17], Mixup [77], Cutmix [75], random erasing [82] and stochastic depth [35], but not repeated augmentation [31] and Exponential Moving Average (EMA) [45] which do not enhance performance.”
- Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement (2023, methodology): “We consider the following aug-mentations: Random-Resize-Crop ( RRC ), MixUp [72] and CutMix [70] ( Mixing ), and RandomAugment [14] and Ran-domErase ( RA/RE ).”
- Unified Contrastive Learning in Image-Text-Label Space (2022, methodology): “When over-fitting is less severe, such as training on larger datasets (from CIFAR to ImageNet) or with strong augmentation (MixUp [71] and CutMix [69]), our method is still on par with CE.”
- Occluded Video Instance Segmentation: A Benchmark (2021, background): “Someworks (DeVries &Taylor, 2017; Yun et al., 2019; Dwibedi et al., 2017; Ghiasi et al., 2021) have proposed augmenting the common datasets (e.”