Multi-Label Image Recognition With Graph Convolutional Networks

By Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, Yanwen Guo. 2019

1,230 citations recorded by the source corpus.

The task of multi-label image recognition is to predict a set of object labels that present in an image. As objects normally co-occur in an image, it is desirable to model the label dependencies to improve the recognition performance. To capture and explore such important dependencies, we propose a multi-label classification model based on Graph Convolutional Network (GCN). The model builds a directed graph over the object labels, where each node (label) is represented by word embeddings of a label, and GCN is learned to map this label graph into a set of inter-dependent object classifiers. These classifiers are applied to the image descriptors extracted by another sub-net, enabling the whole network to be end-to-end trainable. Furthermore, we propose a novel re-weighted scheme to create an effective label correlation matrix to guide information propagation among the nodes in GCN. Experiments on two multi-label image recognition datasets show that our approach obviously outperforms other existing state-of-the-art methods. In addition, visualization analyses reveal that the classifiers learned by our model maintain meaningful semantic topology.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Graph Learning

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, background): “Alternative solutions to capture label co-occurrences include learning joint input-label embeddings with rankingbased losses [66, 31, 70, 29, 15], graph neural networks [8], and using loss functions that directly account for those [15, 63, 42, 54].”
Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, methodology): “In recent multi-label image classification [65,67,42,68,26], it is a common practice to fine-tune a pre-trained model to a target dataset.”
Asymmetric Loss For Multi-Label Classification (2020, background): “Notable success was reported by exploiting label correlation via graph neural networks which represent the label relationships [6, 5, 11] or word embeddings based on knowledge priors [6, 32].”
Mt-Gcn For Multi-Label Audio Tagging With Noisy Labels (2020, methodology): “For more details we refer interested readers to [13].”
BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation (2021, background): “Graph neural networks (GNNs) have received extensive attention from researchers due to their excellent performance on various graph learning tasks such as social analysis [24, 17, 29], drug discovery [12, 25], traffic forecasting [18, 3, 6], recommendation system [38, 32] and computer vision [39, 4]”
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets (2020, methodology): “There’s also a popular trend to model label correlation with graph structure [21,4].”
Evolutionary Architecture Search for Graph Neural Networks (2020, background): “Due to the encouraging learning ability for graph-structured data in many domains, GNN has recently seen a plethora of successful realworld applications such as image recognition [5], new drug discovery [6], and traffic prediction [7], ect.”
The Emerging Trends of Multi-Label Learning (2020, background): “Beginners can try representative deep works of advanced RNN [152], GCN [40], [157] to address MLC problems.”
General Multi-label Image Classification with Transformers (2020, methodology): “Referencing previous works [9, 8, 7], we employ several metrics to evaluate the proposed method and existing meth-ods.”
HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs (2018, background): “…vertices in each edge / hyperedge, implicit regularisation of graph convolutional networks (GCNs) [25] avoids this restriction and enables application to a broader range of problems in combinatorial optimisation [19, 26, 38, 31], computer vision [12, 37], natural language processing [44, 34], etc.”
AGCN: Augmented Graph Convolutional Network for Lifelong Multi-Label Image Recognition (2022, positive): “Furthermore, some methods [9, 20] built the label relationships using the graph structure and used graph convolutional network (GCN) to enhance the representation.”