Deep Collaborative Embedding for Social Image Understanding

By Zechao Li, Jinhui Tang, Tao Mei. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

347 citations recorded by the source corpus.

In this work, we investigate the problem of learning knowledge from the massive community-contributed images with rich weakly-supervised context information, which can benefit multiple image understanding tasks simultaneously, such as social image tag refinement and assignment, content-based image retrieval, tag-based image retrieval and tag expansion. Towards this end, we propose a Deep Collaborative Embedding (DCE) model to uncover a unified latent space for images and tags. The proposed method incorporates the end-to-end learning and collaborative factor analysis in one unified framework for the optimal compatibility of representation learning and latent space discovery. A nonnegative and discrete refined tagging matrix is learned to guide the end-to-end learning. To collaboratively explore the rich context information of social images, the proposed method integrates the weakly-supervised image-tag correlation, image correlation and tag correlation simultaneously and seamlessly. The proposed model is also extended to embed new tags in the uncovered space. To verify the effectiveness of the proposed method, extensive experiments are conducted on two widely-used social image benchmarks for multiple social image understanding tasks. The encouraging performance of the proposed method over the state-of-the-art approaches demonstrates its superiority.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/69adf188e4c7d863edd3a4c0b300d2e9f35ae8e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yi Yang
2. Liang Zheng
3. Dacheng Tao
4. Xuanyi Dong
5. Jinhui Tang
6. Yutian Lin
7. Philip S. Yu
8. Yan Yan

Citations in the corpus

• Few-Shot Image Recognition With Knowledge Transfer (2019, background): “Recently classical deep learning models have achieved remarkable success on many computer vision and image understanding tasks [11, 7, 16, 27, 15].”
• A Bottom-Up Clustering Approach to Unsupervised Person Re-Identification (2019, background): “In recent years, some transfer learning methods (Hehe et al. 2018; Peng et al. 2016; Deng et al. 2018) are proposed upon the success of deep learning (Dong et al. 2018a; Li, Tang, and Mei 2018).”
• Towards Non-I.I.D. image classification: A dataset and baselines (2019, methodology): “NUSWIDE dataset and its extended version [14] focus on social image understanding including the tasks of tag completion, image retrieval and so on, launching the deep models like DCB [15] , WDMF [16] and WDML [17] .”
• Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey (2017, methodology): “In addition, we can combine Topic models with matrix factorization methods to image understanding, tag assignment and semantic discovery from social image datasets [84, 85].”
• A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning (2021, background): “[152] present a novel deep collaborative embedding (DCE) model, which learns knowledge from weakly supervised community-contributed resources, for multiple image understanding tasks simultaneously.”