Reinforcement Knowledge Graph Reasoning for Explainable Recommendation

By Yikun Xian, Zuohui Fu, S. Muthu Muthukrishnan, Gerard de Melo, Yongfeng Zhang. 2019

562 citations recorded by the source corpus.

Recent advances in personalized recommendation have sparked great interest in the exploitation of rich structured information provided by knowledge graphs. Unlike most existing approaches that only focus on leveraging knowledge graphs for more accurate recommendation, we aim to conduct explicit reasoning with knowledge for decision making so that the recommendations are generated and supported by an interpretable causal inference procedure. To this end, we propose a method called Policy-Guided Path Reasoning (PGPR), which couples recommendation and interpretability by providing actual paths in a knowledge graph. Our contributions include four aspects. We first highlight the significance of incorporating knowledge graphs into recommendation to formally define and interpret the reasoning process. Second, we propose a reinforcement learning (RL) approach featured by an innovative soft reward strategy, user-conditional action pruning and a multi-hop scoring function. Third, we design a policy-guided graph search algorithm to efficiently and effectively sample reasoning paths for recommendation. Finally, we extensively evaluate our method on several large-scale real-world benchmark datasets, obtaining favorable results compared with state-of-the-art methods.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Yongfeng Zhang
5. Xu Chen
6. Meng Wang
7. Yong Li
8. Chen Gao

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Yongfeng Zhang
4. Tat-seng Chua
5. Xu Chen
6. Meng Wang
7. Xing Xie
8. Yong Li

Citations in the corpus

• Explainable Recommendation: A Survey and New Perspectives (2018, background): “The trained policy is then adopted for the path reasoning phase to make recommendations to the user, while the path serves the explanation for a recommendation (Xian et al., 2019).”
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, methodology): “However, these meta-path-based solutions also face some challenges in terms of how to obtain these meta-paths [121, 211].”
• Knowledge Graphs: Opportunities and Challenges (2023, methodology): “Neural network-based knowledge reasoning method utilizes neural networks to infer new triplets given the body of knowledge in the graph (Xian et al. 2019).”
• A Survey on Knowledge Graph-Based Recommender Systems (2020, methodology): “Recently, Xian et al. [85] proposed Policy-Guided Path Reasoning (PGPR) to use reinforcement learning (RL) to search for reasonable paths between user-item pairs.”
• A Survey of Knowledge-enhanced Text Generation (2020, background): “An intelligent machine is supposed to have the ability to conduct explicit reasoning over relational paths to make multiple inter-related decisions, rather than merely embedding entities in KGs as latent vectors [157, 160].”
• Knowledge Graph Contrastive Learning for Recommendation (2022, methodology): “To overcome such data sparsity problem, Knowledge Graphs (KGs) serving as useful external sources have been incorporated into the recommender system to enhance the user and item representation process, by encoding additional item-wise semantic relatedness [15, 39, 47].”
• A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, background): “While explicitly modeling high-order connectivity, it is highly challenging in real-world recommendation scenarios because most of these methods require extensive domain knowledge to define meta-paths or labor-intensive feature engineering to obtain qualified paths [167], [170].”
• Learning Intents behind Interactions with Knowledge Graph for Recommendation (2021, background): “GNN-based Methods [17, 38, 39, 41, 47] are founded upon the information aggregation mechanism of graph neural networks (GNNs) [13, 14, 19, 34, 42].”
• A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “PGPR [207] performs reinforcement policy-guided path reasoning over knowledge-graph-based user-item interaction.”
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning (2021, background): “image classification [46] and knowledge graph [47], [48].”
• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022, background): “Besides classical rating prediction and direct user-item matching-based recommendation for users, recent works are broadening the spectrum to new application scenarios such as sequential recommendation [20, 56, 59, 76], conversational recommendation [7, 57, 72], explainable recommendation [29, 58, 6”
• Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Recently, researchers incorporate deep reinforcement learning (DRL)models into interactive recommender systems [213, 20, 190, 215, 65, 230, 24, 68, 97, 130, 225, 231, 175].”
• Reinforcement Learning based Recommender Systems: A Survey (2021, methodology): “(a) MADDPG architecture, (b) MASSA architecture, (c) Graph-based reasoning example [132]”