Meta-Learning to Detect Rare Objects

By Yu-Xiong Wang, Deva Ramanan, Martial Hebert. 2019

343 citations recorded by the source corpus.

Few-shot learning, i.e., learning novel concepts from few examples, is fundamental to practical visual recognition systems. While most of existing work has focused on few-shot classification, we make a step towards few-shot object detection, a more challenging yet under-explored task. We develop a conceptually simple but powerful meta-learning based framework that simultaneously tackles few-shot classification and few-shot localization in a unified, coherent way. This framework leverages meta-level knowledge about "model parameter generation" from base classes with abundant data to facilitate the generation of a detector for novel classes. Our key insight is to disentangle the learning of category-agnostic and category-specific components in a CNN based detection model. In particular, we introduce a weight prediction meta-model that enables predicting the parameters of category-specific components from few examples. We systematically benchmark the performance of modern detectors in the small-sample size regime. Experiments in a variety of realistic scenarios, including within-domain, cross-domain, and long-tailed settings, demonstrate the effectiveness and generality of our approach under different notions of novel classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Grounded Language-Image Pre-training (2021, methodology): “We vary the amount of task-specific annotated data, from zero-shot (no data provided), to X-shot (providing at least X examples per category [22, 57, 61]), to using all data in the training set.”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “This line of research is also related to few-shot [10, 31, 67, 68, 71] and zero-shot detection [6, 20, 43, 70] methods, which attempt to detect novel classes given only a few samples or class descriptions.”
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Other more complicated learning methods ( e.g ., meta-learning [63, 29, 32], causal inference [58], Bayesian methods [34], etc .) are also used to deal with long-tail data.”
Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, neutral): “Meta-learning [51] focuses on learning lots of related tasks to acquire common knowledge adaptable to new unseen tasks, and has achieved great success in many computer vision problems [26,61,27].”
Incremental Object Detection via Meta-Learning (2020, neutral): “[30] learn a meta-model that predicts the weights of the RoI Head, that is finally fine-tuned.”
Frustratingly Simple Few-Shot Object Detection (2020, methodology): “We describe the existing meta-learning based few-shot object detection networks, including FSRW (Kang et al., 2019), Meta R-CNN (Yan et al., 2019) and MetaDet (Wang et al., 2019b), in this section to draw comparisons with our approach.”
Learning to Segment the Tail (2020, background): “On the other hand, conventional few-shot learning that transfers the model from a data-rich “base set” to a data-poor “novel set” [39, 43], however, is not yet practical in LVIS, as any base or novel split will be eventually imbalanced due to the scale, undermining the generalization ability that is”
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning (2022, unknown): “…& Yu, 2017; Nichol & Schulman, 2018; Snell et al., 2017; Wang et al., 2016; Li & Malik, 2017; Vinyals et al., 2016; Andrychowicz et al., 2016; Brock et al., 2018; Zintgraf et al., 2019a; Wang et al., 2019; Achille et al., 2019; Li et al., 2018a; Hsu et al., 2018; Obamuyide & Vlachos, 2019).”
Learning a Universal Template for Few-shot Dataset Generalization (2021, background): “Keeping with the spirit of separating shared from specific weights, (Wang et al., 2019) design a model for few-shot object detection based on the insight of disentangling category-specific and category-agnostic parameters, in a similar spirit to FLUTE’s dataset-specific and dataset-agnostic paramete”
3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding (2022, background): “Research does not only cover the advancement of performance under classic learning paradigms (supervised, offline training) [20, 21, 22, 23, 24, 25, 26, 27], but also encompasses areas such as weakly-supervised learning [28, 29, 30, 31], few-shot learning [32, 33, 34, 35, 36, 37, 38], and semi-super”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “Consistent with the few-shot learning setup in [23, 59, 64], we use VOC 07 and 12 train-val sets for training and VOC 07 test set for testing.”
Boosting Long-tailed Object Detection via Step-wise Learning on Smooth-tail Data (2023, background): “Approaches of few-shot object detection can be categorized into meta-learning based [34, 11, 36, 38] and fine-tuning based methods [33, 35, 27].”
FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (2021, methodology): “In few-shot detection, one might naturally conjecture the localization of novel objects is going to under-perform its base categories counterpart, with the concern that rare objects would be deemed as background [14, 13, 18].”