Mask R-CNN
By Kaiming He, Georgia Gkioxari, Piotr Doll'ar, Ross B. Girshick. 2017
32,519 citations recorded by the source corpus.
We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without tricks, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code will be made available.
Subfields: Computer Vision and Pattern Recognition, Model Architectures
View Paper: https://arxiv.org/abs/1703.06870
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Yi Yang
In Model Architectures
- Quoc V. Le
- Alan L. Yuille
- Jianfeng Gao
- Yoshua Bengio
- Jiashi Feng
- Kaiming He
- Dacheng Tao
- Christoph Feichtenhofer
Citations in the corpus
- Masked Autoencoders Are Scalable Vision Learners (2021, methodology): “We fine-tune Mask R-CNN [24] end-to-end on COCO [37].”
- Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “The detector is Faster R-CNN [52] with a backbone of R50-dilated-C5 or R50-C4 [32] (details in appendix), with BN tuned, implemented in [60].”
- Rethinking ImageNet Pre-Training (2018, methodology): “As modern object detectors [9, 15, 8, 36, 35, 28, 26, 13] evolved under the pre-training paradigm, the belief that training from scratch is non-trivial became conventional wisdom.”
- Exploring the Limits of Weakly Supervised Pretraining (2018, methodology): “We use Mask R-CNN [4,36] and experiment with ResNeXt101 FPN [37] backbones of three different capacities (see Figure 8).”
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “Higher resolutions, such as 600x600, are also widely used in object detection ConvNets (He et al., 2017; Lin et al., 2017).”
- ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks (2019, methodology): “Thus we directly use the bounding box proposals provided by [33], which use a Mask R-CNN [34] pretrained on the COCO dataset.”
- Learning Representations by Maximizing Mutual Information Across Views (2019, methodology): “Similarly, large convolutional networks pre-trained on large supervised corpora have been widely used to improve performance across the spectrum of computer vision tasks [Donahue et al., 2014, Ren et al., 2015, He et al., 2017, Carreira and Zisserman, 2017].”
- Weight Standardization (2019, methodology): “This enables ResNeXt to train with GN in Mask R-CNN [10] and Faster R-CNN [34], which we will discuss in the next subsection.”
- CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “For instance segmentation, we take the same training settings as that of Mask-RCNN [16].”
- Relation Networks for Object Detection (2017, methodology): “The modern CNN based methods mostly have a simple regular network structure [25, 23].”
- Non-local Neural Networks (2017, methodology): “We experiment on the Mask R-CNN baseline [19] for COCO [33] object detection/segmentation and human pose estimation (keypoint detection).”
- VirTex: Learning Visual Representations from Textual Annotations (2020, methodology): “Visual Backbone Ablations: Bigger visual backbones tend to show improvements on many vision tasks [2, 9, 103].”
- The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes (2020, methodology): “We do the same analysis but this time for bounding box labels from Mask R-CNN [29].”
- Natural Adversarial Examples (2019, background): “Research on the ImageNet benchmark has led to numerous advances in classification [30], object detection [27], segmentation [19], and perceptual metrics [47].”
- Benchmarking Detection Transfer Learning with Vision Transformers (2021, methodology): “…role in the study of transfer learning over the last decade ( e.g ., [35, 14, 9, 17]), was not explored in the pioneering work on ViT training [10, 7, 5]—supervised or unsupervised—due to the challenges (described shortly) of integrating ViTs into common detection models, like Mask R-CNN [19].”
- Early Convolutions Help Transformers See Better (2021, background): “These architectures are now core components in object detection ( e.g ., [34]), instance segmentation ( e.g ., [18]), and semantic segmentation ( e.g ., [26]).”
- Self-supervised Pretraining of Visual Features in the Wild (2021, methodology): “We train a Mask-RCNN model [23] on the COCO benchmark [33] with pretrained RegNetY-64GF and RegNetY-128GF as backbones.”
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour (2017, methodology): “For these experiments, we initialize Mask R-CNN from the released MSRA ResNet-50 model, as was done in [14].”