Data Augmentation Using Random Image Cropping and Patching for Deep CNNs

By Ryo Takahashi, Takashi Matsubara, Kuniaki Uehara. IEEE transactions on circuits and systems for video technology (Print), 2018

406 citations recorded by the source corpus.

Deep convolutional neural networks (CNNs) have achieved remarkable results in image processing tasks. However, their high expression ability risks overfitting. Consequently, data augmentation techniques have been proposed to prevent overfitting while enriching datasets. Recent CNN architectures with more parameters are rendering traditional data augmentation techniques insufficient. In this study, we propose a new data augmentation technique called random image cropping and patching (RICAP) which randomly crops four images and patches them to create a new training image. Moreover, RICAP mixes the class labels of the four images, resulting in an advantage of the soft labels. We evaluated RICAP with current state-of-the-art CNNs (e.g., the shake-shake regularization model) by comparison with competitive data augmentation techniques such as cutout and mixup. RICAP achieves a new state-of-the-art test error of 2.19% on CIFAR-10. We also confirmed that deep CNNs with RICAP achieve better results on classification tasks using CIFAR-100 and ImageNet, an image-caption retrieval task using Microsoft COCO, and other computer vision tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1f5066018662b7c7d13a57611e6f118b2871d39f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Deep Learning for Scientific Discovery (2020, methodology): “Inspired by these simple methods are two very successful image augmentation strategies, cutout [44], which removes a patch from the input image, and RICAP [222], which combines patches from four diﬀerent input image to create a new image (with new label a combination of the original labels.)”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, background): “Rather than occluding a portion of an image, CutMix replaces a portion of an image with a portion of a different image (Yun et al., 2019; Takahashi et al., 2019).”
A review of medical image data augmentation techniques for deep learning applications (2021, methodology): “also utilised Random Image Cropping And Patching (RICAP), where, rather than blending the images, four images are randomly selected, cropped and then patched together.(89,110) Unlike other augmentation techniques, basic augmentation methods do not aim to produce ‘realistic’ images.”
Adversarial Robustness with Semi-Infinite Constrained Learning (2021, background): “…choices of distributions lead to other robust learning methods involving random removal of patches from the image (e.g., Cutout [86, 87]), random replacement of patches (e.g., CutMix [88, 89]), or arbitrary generative models, i.e., δ ∼ G # N ( 0 , σ 2 I ) for some measurable G : z → R d [85].”
Enhancing Performance of Deep Learning Models with different Data Augmentation Techniques: A Survey (2020, background): “The random cropping and translation are different in the sense that cropping will decrease the size of the image while translations preserve its spacial dimensions [12] [18].”
FMix: Enhancing Mixed Sample Data Augmentation (2020, background): “Note that we view current masking MSDAs as equivalent for the purpose of our analysis since they all fundamentally mix rectangular regions [39, 40, 51, 29].”