Learning Compositional Representations for Few-Shot Recognition

By Pavel Tokmakov, Yu-Xiong Wang, Martial Hebert. 2018

134 citations recorded by the source corpus.

One of the key limitations of modern deep learning approaches lies in the amount of data required to train them. Humans, by contrast, can learn to recognize novel categories from just a few examples. Instrumental to this rapid learning ability is the compositional structure of concept representations in the human brain --- something that deep learning models are lacking. In this work, we make a step towards bridging this gap between human and machine learning by introducing a simple regularization technique that allows the learned representation to be decomposable into parts. Our method uses category-level attribute annotations to disentangle the feature space of a network into subspaces corresponding to the attributes. These attributes can be either purely visual, like object parts, or more abstract, like openness and symmetry. We demonstrate the value of compositional representations on three datasets: CUB-200-2011, SUN397, and ImageNet, and show that they require fewer examples to learn classifiers for novel categories.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

Shaping Visual Representations with Language for Few-Shot Classification (2019, neutral): “In this framework, learning with attributes and other domain-speciﬁc rationales has been tackled extensively [8, 9, 29], but language remains relatively unexplored.”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Discriminative parts [5, 10, 22, 34, 42, 83] and visual words [74, 100] have a rich history, and have found applications in deformable-parts models [26], and classification [70, 100].”
Incremental Few-Shot Object Detection (2020, neutral): “Note that there are many more recent FSL methods [6, 26, 7, 17, 47, 55, 50, 27, 39, 11, 16] that produce better performance on image classification tasks than MAML.”
Compositional Few-Shot Recognition with Primitive Discovery and Enhancing (2020, methodology): “e self-supervision from object split orders to facilitate the discovery of part-related primitives and reduce the effect of semantic gaps. Specifically, as visual primitives can refer to object parts [51], the training procedure should encourage the model to recognize object parts. However, as curre”
A survey of deep meta-learning (2020, background): “, cn(x), which in turn could help the performance in low-data regimes (Tokmakov et al., 2019).”
Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition (2020, unknown): “But for zebra’s unique character, zebra stripes, it is hard to transfer high-level patterns (e.g., semantic parts [37]) from dogs, but it is much easier to transfer mid-level patterns such as stripe itself to describe it.”
Locality and compositionality in zero-shot learning (2019, background): “Speciﬁcally, we explore the role of compositionality and locality (Tokmakov et al., 2018; Stone et al., 2017) as two principles that lead to good generalization.”
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness (2021, unknown): “Representative works include [56, 51, 19, 20, 22, 48, 7, 41, 6, 40, 36, 60, 45, 52, 64], etc.”
Adaptive Cross-Modal Few-Shot Learning (2019, background): “One concurrent work (Tokmakov et al., 2018) also leverages side information to help few-shot learning.”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, background): “g object masks for large-scale datasets is laborious, expensive, and can be impractical [9,10,11]. It is also not consistent with cognitive learning, which can build a model upon few-shot supervision [12]. Given a few examples, termed support images, and the related segmentation masks [13], few-shot”
Meta-Learning to Detect Rare Objects (2019, background): “Few-shot learning is a fundamental yet unsolved problem in machine learning and computer vision [61, 15, 30, 32, 67, 23, 20, 64].”
Classification-by-Components: Probabilistic Modeling of Reasoning over a Set of Components (2019, background): “In [18], a form of reasoning is introduced similar to the indeﬁnite reasoning state by occluding parts of the learned representation.”
Contextualizing Multiple Tasks via Learning to Decompose (2021, methodology): “[38, 54, 68, 58, 59] adapts the feature embeddings or the model initialization given a few-shot support set, [48, 57] compose instance representations with the supervision of attributes, and [4] utilizes the ﬁne-grained visual concepts.”
Benchmark for Compositional Text-to-Image Synthesis (2021, background): “A typical example is: given images of a “young tiger” and an “old car”, is the model able to recognize an “old tiger”? In visual recognition, researchers explored the compositionality of objects and attributes so that the model can generalize to unseen combinations of visual concepts [28, 29, 53, 50”
Mutual CRF-GNN for Few-shot Learning (2021, background): “(1) Metric learning based methods [52, 45, 47, 53, 9, 58, 42, 51] focus on obtaining a generalizable encoder to transform all samples into a common metric space and then use the distances between query features and support features to perform classification.”
Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations (2020, methodology): “For FSL, Tokmakov et al. (2018) introduced an explicit penalty to encourage compositionality when dealing with attribute representations of classes.”