Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

By Golnaz Ghiasi, Yin Cui, A. Srinivas, Tsung-Yi Lin, Ekin Dogus Cubuk, Quoc V. Le, Barret Zoph. 2020

1,320 citations recorded by the source corpus.

Building instance segmentation models that are data-efficient and can handle rare object categories is an important challenge in computer vision. Leveraging data augmentations is a promising direction towards addressing this challenge. Here, we perform a systematic study of the Copy-Paste augmentation (e.g., [13], [12]) for instance segmentation where we randomly paste objects onto an image. Prior studies on Copy-Paste relied on modeling the surrounding visual context for pasting the objects. However, we find that the simple mechanism of pasting objects randomly is good enough and can provide solid gains on top of strong baselines. Furthermore, we show Copy-Paste is additive with semi-supervised methods that leverage extra data through pseudo labeling (e.g. self-training). On COCO instance segmentation, we achieve 49.1 mask AP and 57.3 box AP, an improvement of +0.6 mask AP and +1.5 box AP over the previous state-of-the-art. We further demonstrate that Copy-Paste can lead to significant improvements on the LVIS benchmark. Our baseline model outperforms the LVIS 2020 Challenge winning entry by +3.6 mask AP on rare categories.1

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, methodology): “Creating synthetic datasets by inserting objects onto backgrounds has been used for training [75, 16, 21] and evaluating models [36], but previous works do not systematically vary object size, location or orientation, or analyze translation and rotation robustness only at the image level [18].”
Benchmarking Detection Transfer Learning with Vision Transformers (2021, background): “…formula was developed in [18], which demonstrated good from-scratch performance when training with normalization layers and for long enough, and [13], which demonstrated that a simple data augmentation method called large-scale jitter (LSJ) is effective at preventing overﬁtting and improves…”
A Survey of Data Augmentation Approaches for NLP (2021, neutral): “, 2019), CUTOUT (DeVries and Taylor, 2017) and COPY-PASTE (Ghiasi et al., 2020).”
Segment Anything (2023, methodology): “…on manual and semi-automatic data only, it was trained for longer (177656 iterations instead of 90k) with large-scale jitter data augmentation [40], simulated interactive training used only point and mask prompts (no boxes) and sampled only 4 points per mask during training (reducing from our…”
Pix2seq: A Language Modeling Framework for Object Detection (2021, methodology): “For image augmentation during training, we perform scale jittering with random crops (Ghiasi et al., 2021; Wu et al., 2019) with strength of [0 .”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, neutral): “7 box AP (Copy-paste [23] without external data) and +2.”
RegionCLIP: Region-based Language-Image Pretraining (2021, methodology): “CLIP [37] CLIP400M ViT-B/32 ViLD* [18] RN50-FPN 16x+LSJ [15] Base 16.”
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “Copying-and-pasting foreground instances at different locations into different background scenes has shown to be a simple and effective data augmentation technique for object detection and instance segmentation, as shown in [9,13,82].”
Exploring Plain Vision Transformer Backbones for Object Detection (2022, methodology): “The input image is 1024×1024, augmented with large-scale jittering [18] during training.”
Instance Segmentation for Autonomous Log Grasping in Forestry Operations (2022, background): “Improved data augmentation, such as copy-paste [32] could also be employed to reduce dependency on labeled images.”
Masked-attention Mask Transformer for Universal Image Segmentation (2021, methodology): “With ResNet [25] backbone, Mask2Former outperforms a strong Mask R-CNN [24] baseline using large-scale jittering (LSJ) augmentation [18, 23] while requiring 8 × fewer training iterations.”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “Large vocabulary detection has gained much attention lately in research community [22, 24, 52, 65] because the problem captures the Zipf’s law of visual world, i.”
Occluded Video Instance Segmentation: A Benchmark (2021, background): “Someworks (DeVries &Taylor, 2017; Yun et al., 2019; Dwibedi et al., 2017; Ghiasi et al., 2021) have proposed augmenting the common datasets (e.”
Bottleneck Transformers for Visual Recognition (2021, background): “One way to address this is to increase the amount of multi-scale jitter which has been known to improve the performance of detection and segmentation systems [16, 18].”
Simple Open-Vocabulary Object Detection with Vision Transformers (2022, methodology): “Additionally, we use image scale augmentation similar to “large scale jitter” [11].”