Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild

By Yang Xiao, Vincent Lepetit, Renaud Marlet. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

421 citations recorded by the source corpus.

Detecting objects and estimating their viewpoints in images are key tasks of 3D scene understanding. Recent approaches have achieved excellent results on very large benchmarks for object detection and viewpoint estimation. However, performances are still lagging behind for novel object categories with few samples. In this paper, we tackle the problems of few-shot object detection and few-shot viewpoint estimation. We demonstrate on both tasks the benefits of guiding the network prediction with class-representative features extracted from data in different modalities: image patches for object detection, and aligned 3D models for viewpoint estimation. Despite its simplicity, our method outperforms state-of-the-art methods by a large margin on a range of datasets, including PASCAL and COCO for few-shot object detection, and Pascal3D+ and ObjectNet3D for few-shot viewpoint estimation. Furthermore, when the 3D model is not available, we introduce a simple category-agnostic viewpoint estimation method by exploiting geometrical similarities and consistent pose labeling across different classes. While it moderately reduces performance, this approach still obtains better results than previous methods in this setting. Last, for the first time, we tackle the combination of both few-shot tasks, on three challenging benchmarks for viewpoint estimation in the wild, ObjectNet3D, Pascal3D+ and Pix3D, showing very promising results.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3d30266c67d512a2c08b3a17eee6ba6c59d81c4a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Daniel Cohen-Or

In Artificial Intelligence

1. Lifu Huang
2. Rinon Gal
3. Daniel Cohen-Or
4. Bo Sun
5. Yotam Nitzan
6. Xi Qiu
7. Yuxuan Zhao
8. Jianan Wu

In Computer Vision and Pattern Recognition

1. Rinon Gal
2. Daniel Cohen-Or
3. Yotam Nitzan
4. Ofir Brenner
5. Bo Sun
6. Xi Qiu
7. Yuxuan Zhao
8. Jianan Wu

Citations in the corpus

• LARGE: Latent-Based Regression through GAN Semantics (2021, background): “In particular, this scarcity holds for regression tasks, where a majority of research focused on pose estimation using strong task-specific priors [64] or applying meta-learning ideas to keypoint extraction and gaze estimation frameworks [36, 55].”
• The Art of Prompting: Event Detection based on Type Specific Prompts (2022, background): “Few-shot ED: Two primary learning strategies in few-shot classification tasks are MetaLearning (Kang et al., 2019; Li et al., 2021; Xiao and Marlet, 2020; Yan et al., 2019; Chowdhury et al., 2021) and Metric Learning (Sun et al.”
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, neutral): “7AP ✓ FsDetView [250] ECCV’20 R-CNN Meta ✓ ✓ ✓ 12.”
• FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (2021, methodology): “We strictly follow the consistent few-shot detection data construction and evaluation protocol [43, 16, 17, 15] to ensure fair and direct comparison.”
• DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2021, methodology): “As one of the research branches, fewshot object detection (FSOD) is a much more challenging task than both few-shot classification and object detection [5, 19, 46, 51, 53].”