MMGCN: Multi-modal Graph Convolution Network for Personalized Recommendation of Micro-video

By Yin-wei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, Tat-seng Chua. ACM Multimedia, 2019

866 citations recorded by the source corpus.

Personalized recommendation plays a central role in many online content sharing platforms. To provide quality micro-video recommendation service, it is of crucial importance to consider the interactions between users and items (i.e. micro-videos) as well as the item contents from various modalities (e.g. visual, acoustic, and textual). Existing works on multimedia recommendation largely exploit multi-modal contents to enrich item representations, while less effort is made to leverage information interchange between users and items to enhance user representations and further capture user's fine-grained preferences on different modalities. In this paper, we propose to exploit user-item interactions to guide the representation learning in each modality, and further personalized micro-video recommendation. We design a Multi-modal Graph Convolution Network (MMGCN) framework built upon the message-passing idea of graph neural networks, which can yield modal-specific representations of users and micro-videos to better capture user preferences. Specifically, we construct a user-item bipartite graph in each modality, and enrich the representation of each node with the topological structure and features of its neighbors. Through extensive experiments on three publicly available datasets, Tiktok, Kwai, and MovieLens, we demonstrate that our proposed model is able to significantly outperform state-of-the-art multi-modal recommendation methods.

Subfields: Computer Vision and Pattern Recognition, Information Systems

View Paper: https://www.semanticscholar.org/paper/75d6b55e8b9146e3d9dce9091cfb03211a664b3a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Liqiang Nie
5. Meng Wang
6. Chao Huang
7. Yin-wei Wei
8. Yongdong Zhang

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Liqiang Nie
5. Meng Wang
6. Chao Huang
7. Fuli Feng
8. Yin-wei Wei

Citations in the corpus

• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation (2020, background): “For example, a recent trend is to exploit auxiliary information such as item knowledge graph [35, 38], social network [41] and multimedia content [45] for recommendation, where GCNs have set up the new state-of-the-art.”
• Mining Latent Structures for Multimedia Recommendation (2021, background): “• MMGCN [38] is one of the state-of-the-art multimodal recommendation methods, which constructs modal-specific graphs and refines modal-specific representations for users and items.”
• Dynamic Modality Interaction Modeling for Image-Text Retrieval (2021, background): “It benefits a variety of applications, ranging from cross-modal retrieval [11, 12, 23, 41] tomultimedia recommendation [36].”
• LLMRec: Large Language Models with Graph Augmentation for Recommendation (2023, methodology): “In contrast, approaches rely on limited content, such as VBPR [11] using only visual features, or CF-based architectures like NGCF [32], without side information, yield significantly diminished results.”
• Graph Neural Networks in Recommender Systems: A Survey (2020, background): “For instance, MMGCN [165] constructs a user-item bipartite graph for each modality and applies GNN to propagate information for each graph, respectively.”
• Combining Graph Neural Networks With Expert Knowledge for Smart Contract Vulnerability Detection (2021, methodology): “[39], [40] and develops a layer-wise propagation network using the Laplacian matrix, which achieves promising performance on graph node classification tasks.”
• Multi-modal Knowledge Graphs for Recommender Systems (2020, background): “We compare our proposed MKGAT model with some state-of-the-art baselines, which include FM-based Method (NFM), KG-basedmethod (CKE, KGAT),Multi-modalmethod (MMGCN).”
• Contrastive Learning for Cold-Start Recommendation (2021, background): “, clicks, rates, views) is central to modern CF models [14, 15, 36, 48].”
• Graph Retrieval-Augmented Generation: A Survey (2024, neutral): “This section will delve into the prevalent obstacles and outline prospective avenues for future research in the field of GraphRAG.”
• Lipschitz Continuity Guided Knowledge Distillation (2021, background): “Recently, deep learning models have driven great advances in computer vision [13, 8], natural language process [36, 33], information retrieval [42, 43] and multimodal modelling [19, 18].”
• Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI (2021, background): “Wei et al. (2019) [27] proposed a MM Graph Convolutional Network in order to consider the different modalities contained in videos — visual, acoustic, textual — by constructing a user–item interaction graph within each modality, thus learning different user preference functions which eventually…”
• Deep Multimodal Data Fusion (2024, background): “…[42, 182, 182, 208–210], cross-modal retrieval task [30, 204, 217], RGB-depth scene classification task [222], multimodal recommendation task [193, 194], neuroimaging-based disease classification task [158], 3D object localization with natural language descriptions [21], and object segmentation…”