Few-Shot Object Recognition from Machine-Labeled Web Images

By Zhongwen Xu, Linchao Zhu, Yi Yang. 2016

69 citations recorded by the source corpus.

With the tremendous advances made by Convolutional Neural Networks (ConvNets) on object recognition, we can now easily obtain adequately reliable machine-labeled annotations easily from predictions by off-the-shelf ConvNets. In this work, we present an abstraction memory based framework for few-shot learning, building upon machine-labeled image annotations. Our method takes large-scale machine-annotated dataset (e.g., OpenImages) as an external memory bank. In the external memory bank, the information is stored in the memory slots in the form of key-value, in which image feature is regarded as the key and the label embedding serves as the value. When queried by the few-shot examples, our model selects visually similar data from the external memory bank and writes the useful information obtained from related external data into another memory bank, i.e., abstraction memory. Long Short-Term Memory (LSTM) controllers and attention mechanisms are utilized to guarantee the data written to the abstraction memory correlates with the query example. The abstraction memory concentrates information from the external memory bank to make the few-shot recognition effective. In the experiments, we first confirm that our model can learn to conduct few-shot object recognition on clean human-labeled data from the ImageNet dataset. Then, we demonstrate that with our model, machine-labeled image annotations are very effective and abundant resources for performing object recognition on novel categories. Experimental results show that our proposed model with machine-labeled annotations achieves great results, with only a 1% difference in accuracy between the machine-labeled annotations and the human-labeled annotations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Martial Hebert
5. Yu-Xiong Wang
6. Sung Ju Hwang
7. Yi Yang
8. Yu Qiao

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Han-Jia Ye
3. De-chuan Zhan
4. Yi Yang
5. Bernt Schiele
6. Trevor Darrell
7. Yu Qiao
8. Dacheng Tao

Citations in the corpus

• Deep Meta-Learning: Learning to Learn in the Concept Space (2018, background): “Recently, (Xu et al., 2017) show a key-value memory networks for few-shot learning, which is not scalable due to the huge memory size and the heavy cost in memory retrieval.”
• Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, methodology): “Apart from the classification problem, FSL also has a close relationship with other computer vision tasks, e.g., object detection [62], semantic learning [63], video classification [64] and face alignment [65].”
• Generalizing from a Few Examples (2019, background): “The two most popular applications are character recognition [14, 36, 37, 65, 70, 96, 113, 114, 119, 121, 130, 138, 146] and image classification [37, 70, 96, 106, 119, 121, 127, 130, 132, 138, 142, 143, 149].”
• Webly Supervised Learning Meets Zero-shot Learning: A Hybrid Approach for Fine-Grained Classification (2018, unknown): “With the rapid development of deep learning, there are also several CNN approaches on learning from web images for image classification [46, 40, 9, 12, 51, 55].”
• Compound Memory Networks for Few-Shot Video Classification (2018, background): “[37] presented a key-value memory network to facilitate few-shot learning by extracting knowledge from external knowledge bases, e.”
• ProtoGAN: Towards Few Shot Learning for Action Recognition (2019, background): “Learning optimization involves memory based networks like LSTM [26] or memory augmented networks [15] which aim to replace stochastic gradient descent optimizer with the help of external memory.”
• Transductive Propagation Network for Few-shot Learning (2018, background): “mple, children can generalize the concept of “apple” after given a single sample of the apple. This significant gap between human and deep learning reawakens the research interest of few-shot learning [24, 20, 2, 14, 11, 28, 26]. Few-shot learning aims to learn a classifier that can be adapted to acco”
• Adaptive Subspaces for Few-Shot Learning (2020, unknown): “Inspired by such learning abilities, several approaches are developed to study learning from limited samples [3–12].”
• Image Deformation Meta-Networks for One-Shot Learning (2019, background): “Note that since it is purely sampled from Dbase, the newly introduced G does not break the standard one-shot setup as in [34, 22, 5, 17].”
• Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning (2018, background): “This significant gap between human and deep learning has reawakened the research interest on few-shot learning (Vinyals et al., 2016; Snell et al., 2017; Finn et al., 2017; Ravi and Larochelle, 2017; Lee and Choi, 2018; Xu et al., 2017; Wang et al., 2018).”
• LSTD: A Low-Shot Transfer Detector for Object Detection (2018, methodology): “However, the existing low-shot approaches are mainly designed for the standard classi-fication task (Xu, Zhu, and Yang 2017; Hariharan and Gir-shick 2017).”