Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning

By Xiaopeng Yan, Ziliang Chen, Anni Xu, Xiaoxi Wang, Xiaodan Liang, Liang Lin. 2019

610 citations recorded by the source corpus.

Resembling the rapid learning capability of human, low-shot learning empowers vision systems to understand new concepts by training with few samples. Leading approaches derived from meta-learning on images with a single visual object. Obfuscated by a complex background and multiple objects in one image, they are hard to promote the research of low-shot object detection/segmentation. In this work, we present a flexible and general methodology to achieve these tasks. Our work extends Faster /Mask R-CNN by proposing meta-learning over RoI (Region-of-Interest) features instead of a full image feature. This simple spirit disentangles multi-object information merged with the background, without bells and whistles, enabling Faster /Mask R-CNN turn into a meta-learner to achieve the tasks. Specifically, we introduce a Predictor-head Remodeling Network (PRN) that shares its main backbone with Faster /Mask R-CNN. PRN receives images containing low-shot objects with their bounding boxes or masks to infer their class attentive vectors. The vectors take channel-wise soft-attention on RoI features, remodeling those R-CNN predictor heads to detect or segment the objects consistent with the classes these vectors represent. In our experiments, Meta R-CNN yields the new state of the art in low-shot object detection and improves low-shot object segmentation by Mask R-CNN. Code: \url{https://yanxp.github.io/metarcnn.html}.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Grounded Language-Image Pre-training (2021, methodology): “We vary the amount of task-specific annotated data, from zero-shot (no data provided), to X-shot (providing at least X examples per category [22, 57, 61]), to using all data in the training set.”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “This line of research is also related to few-shot [10, 31, 67, 68, 71] and zero-shot detection [6, 20, 43, 70] methods, which attempt to detect novel classes given only a few samples or class descriptions.”
Incremental Object Detection via Meta-Learning (2020, neutral): “[32] and Kang et al .”
Incremental Few-Shot Object Detection (2020, neutral): “The results on COCO in Table 1 show that, while not directly comparable due to using different detection backbones and/or data split, ONCE approaches the performance of the two state-of-the-art models [22, 53].”
Frustratingly Simple Few-Shot Object Detection (2020, methodology): “We describe the existing meta-learning based few-shot object detection networks, including FSRW (Kang et al., 2019), Meta R-CNN (Yan et al., 2019) and MetaDet (Wang et al., 2019b), in this section to draw comparisons with our approach.”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “1 Meta R-CNN* [64] 19.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, neutral): “Meta R-CNN [253] ICCV’19 R-CNN Meta - 12.”
Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector (2019, methodology): “Recent works for few-shot object detection [9, 10, 11, 12] on the other hand all require fine-tuning and thus cannot be directly applied on novel categories.”
Class-Incremental Few-Shot Object Detection (2021, methodology): “We compare LEAST with several state-of-thearts on MS-COCO in Tab.1, which are MetaDet [Wang et al., 2019], Meta-RCNN [Yan et al., 2019], TFA [Wang et al., 2020], Attn-RPN [Fan et al., 2020], and FSDetView [Xiao and Marlet, 2020].”
Multi-Scale Positive Sample Refinement for Few-Shot Object Detection (2020, methodology): “Following the settings in [16,41], object classes are divided into base classes with abundant data and novel classes with only a few training samples.”
The Art of Prompting: Event Detection based on Type Specific Prompts (2022, background): “Few-shot ED: Two primary learning strategies in few-shot classification tasks are MetaLearning (Kang et al., 2019; Li et al., 2021; Xiao and Marlet, 2020; Yan et al., 2019; Chowdhury et al., 2021) and Metric Learning (Sun et al.”
3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding (2022, background): “Research does not only cover the advancement of performance under classic learning paradigms (supervised, offline training) [20, 21, 22, 23, 24, 25, 26, 27], but also encompasses areas such as weakly-supervised learning [28, 29, 30, 31], few-shot learning [32, 33, 34, 35, 36, 37, 38], and semi-super”
DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, methodology): “As one of the research branches, fewshot object detection (FSOD) is a much more challenging task than both few-shot classification and object detection [5, 19, 46, 51, 53].”
Sketch-Guided Object Localization in Natural Images (2020, methodology): “[37] used class-specific attentive vectors inferred from images of objects in a meta-set for applying channel-wise soft attention to feature maps of the proposals.”
FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (2021, methodology): “We visualize the pairwise cosine similarity between class prototypes [19, 20, 21] of a Faster R-CNN box classiﬁer trained with PASCAL VOC [22, 23].”