Improving One-Shot Learning through Fusing Side Information

By Yao-Hung Hubert Tsai, Ruslan Salakhutdinov. 2017

53 citations recorded by the source corpus.

Deep Neural Networks (DNNs) often struggle with one-shot learning where we have only one or a few labeled training examples per category. In this paper, we argue that by using side information, we may compensate the missing information across classes. We introduce two statistical approaches for fusing side information into data representation learning to improve one-shot learning. First, we propose to enforce the statistical dependency between data representations and multiple types of side information. Second, we introduce an attention mechanism to efficiently treat examples belonging to the 'lots-of-examples' classes as quasi-samples (additional training samples) for 'one-example' classes. We empirically show that our learning architecture improves over traditional softmax regression networks as well as state-of-the-art attentional regression networks on one-shot recognition tasks.

Subfields: Artificial Intelligence, Ocean Engineering, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Percy Liang
2. Ruslan Salakhutdinov
3. Mohit Bansal
4. Louis-philippe Morency
5. Trevor Darrell
6. Jianfeng Gao
7. Yejin Choi
8. Luke S. Zettlemoyer

In Ocean Engineering

1. Jun Shu
2. Deyu Meng
3. Zongben Xu
4. Zeynep Akata
5. Sayna Ebrahimi
6. Samarth Sinha
7. Ruslan Salakhutdinov
8. Yao-Hung Hubert Tsai

In Radiology, Nuclear Medicine and Imaging

1. Jun Shu
2. Deyu Meng
3. Zongben Xu
4. Zeynep Akata
5. Trevor Darrell
6. Sayna Ebrahimi
7. Edgar Schönfeld
8. Samarth Sinha

Citations in the corpus

• Learning Rapid-Temporal Adaptations (2017, background): “When a full description consisting of only one or a few examples per label is provided for each task, it resembles one-shot learning [42, 40].”
• Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders (2018, methodology): “The use of auxiliary information was also explored in ReViSE [29], in which a common image-label semantic space for transductive few-shot learning is learned.”
• Learning Factorized Multimodal Representations (2018, background): “…recent works learn visual representations that are maximally dependent with linguistic attributes for improving one-shot image recognition (Tsai & Salakhutdinov, 2017) or introduce tensor product mechanisms to model interactions between the language, visual and acoustic modalities (Liu et…”
• Small Sample Learning in Big Data Era (2018, methodology): “To fuse side information into data representation learning, Tsai and Salakhutdinov (2017) introduced two statistical approaches to improve one-shot learning.”
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, methodology): “Method Venue Type Experimental Dataset Description Query set External labeled set External unlabeled set Congealing [57] CVPR’00 Data Expansion MNIST DAD [62] ECCV’14 Data Expansion Sign Language* FTT [60] CVPR’16 Data Expansion TADB Low-shot-shrink-hallucinate [58] ICCV’17 Data Expansion ImageNet F”