Learning to Segment via Cut-and-Paste
By Tal Remez, Jonathan Huang, Matthew A. Brown. 2018
107 citations recorded by the source corpus.
This paper presents a weakly-supervised approach to object instance segmentation. Starting with known or predicted object bounding boxes, we learn object masks by playing a game of cut-and-paste in an adversarial learning setup. A mask generator takes a detection box and Faster R-CNN features, and constructs a segmentation mask that is used to cut-and-paste the object into a new image location. The discriminator tries to distinguish between real objects, and those cut and pasted via the generator, giving a learning signal that leads to improved object masks. We verify our method experimentally using Cityscapes, COCO, and aerial image datasets, learning to segment objects without ever having seen a mask in training. Our method exceeds the performance of existing weakly supervised methods, without requiring hand-tuned segment proposals, and reaches 90% of supervised performance.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Sunghyun Cho
- Jiwoon Ahn
- Sergey I. Nikolenko
- Suha Kwak
- Lennart Svensson
- Wilhelm Tranheden
- Viktor Olsson
Citations in the corpus
- Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations (2019, methodology): “For instance segmentation, bounding boxes have been widely used as weak labels since they provide every property of objects except shape [24, 44].”
- Synthetic Data for Deep Learning (2019, background): “A related but different field considers unsupervised approaches to segmentation and other computer vision problems based on adversarial architectures, including learning to segment via cut-and-paste [487], unsupervised segmentation by moving objects between pairs of images with inpainting [434], seg”
- Unsupervised Object Segmentation by Redrawing (2019, background): “SEIGAN [42] and Cut & Paste [45] learns to separate object and background by moving the region corresponding to the object to another background and making sure the image is still realistic.”
- ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning (2020, background): “ClassMix also shares similarities with other segmentation-based augmentation strategies [9, 32, 8, 35, 11], where annotated single instances of objects are cut out of images, and pasted onto new background scenes.”