Adversarial Fine-Grained Composition Learning for Unseen Attribute-Object Recognition

By Kun-Juan Wei, Muli Yang, H. Wang, Cheng Deng, Xianglong Liu. 2019

96 citations recorded by the source corpus.

Recognizing unseen attribute-object pairs never appearing in the training data is a challenging task, since an object often refers to a specific entity while an attribute is an abstract semantic description. Besides, attributes are highly correlated to objects, i.e., an attribute tends to describe different visual features of various objects. Existing methods mainly employ two classifiers to recognize attribute and object separately, or simply simulate the composition of attribute and object, which ignore the inherent discrepancy and correlation between them. In this paper, we propose a novel adversarial fine-grained composition learning model for unseen attribute-object pair recognition. Considering their inherent discrepancy, we leverage multi-scale feature integration to capture discriminative fine-grained features from a given image. Besides, we devise a quintuplet loss to depict more accurate correlations between attributes and objects. Adversarial learning is employed to model the discrepancy and correlations among attributes and objects. Extensive experiments on two challenging benchmarks indicate that our method consistently outperforms state-of-the-art competitors by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, neutral): “Thanks to the development of deep learning, like other visual tasks (Wei et al. 2019; Yang et al. 2019; Han et al. 2020; Feng et al. 2020), current instance segmentation methods (He et al. 2017; Bolya et al. 2019; Chen et al. 2020) based on convolutional neural networks (CNNs) have achieved…”
A Variational Autoencoder with Deep Embedding Model for Generalized Zero-Shot Learning (2020, background): “…on building an embedding model that first learns the mapping between visual feature space and semantic space (Akata et al. 2013; Frome et al. 2013; Lampert, Nickisch, and Harmeling 2014; Romera-Paredes and Torr 2015; Reed et al. 2016; Zhang, Xiang, and Gong 2017; Sung et al. 2018; Wei et al. 2019).”
Heterogeneous Graph Attention Network for Unsupervised Multiple-Target Domain Adaptation (2020, unknown): “In particular, adversarial adaptation methods [9] have shown exciting results in UDA, and these methods are analogous to the generative adversarial networks (GANs) [10], where the feature extractor is trained to determine whether the sample comes from source domain or target domain.”
Benchmark for Compositional Text-to-Image Synthesis (2021, background): “A typical example is: given images of a “young tiger” and an “old car”, is the model able to recognize an “old tiger”? In visual recognition, researchers explored the compositionality of objects and attributes so that the model can generalize to unseen combinations of visual concepts [28, 29, 53, 50”
Interpretable Visual Reasoning via Probabilistic Formulation Under Natural Supervision (2020, background): “Since VQA requires high-level understanding of images and the associated questions, visual reasoning is required to provide primitives for deriving a good answer and make VQA model more interpretable for better human understanding [2,26,48,27,58,31,23,36,55].”
Lifelong Zero-Shot Learning (2020, background): “In recent years, Zero-Shot Learning (ZSL) [Socher et al. , 2013; Xian et al. , 2018a; Zhao et al. , 2019; Wei et al. , 2019; Xu et al. , 2019] has gained increasing attention in computer vision [Chang et al. , 2020] and machine learning communities [Yang et al. , 2019].”
Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches (2020, background): “, bounding box [2, 21, 37, 40, 16], to weakly-supervised conditions with only category labels [11, 41, 38, 36, 12, 22, 39, 42].”
The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification (2020, background): “More recent research has therefore concentrated on realizing parts in an unsupervised fashion [7], [9], [23], [32], [36], [43], [49].”