DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning

By Wenhan Xiong, Thi-Lan-Giao Hoang, William Yang Wang. Conference on Empirical Methods in Natural Language Processing, 2017

867 citations recorded by the source corpus.

We study the problem of learning to reason in large scale knowledge graphs (KGs). More specifically, we describe a novel reinforcement learning framework for learning multi-hop relational paths: we use a policy-based agent with continuous states based on knowledge graph embeddings, which reasons in a KG vector-space by sampling the most promising relation to extend its path. In contrast to prior work, our approach includes a reward function that takes the accuracy, diversity, and efficiency into consideration. Experimentally, we show that our proposed method outperforms a path-ranking based algorithm and knowledge graph embedding methods on Freebase and Never-Ending Language Learning datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/857176d022369e963d3ff1be2cb9e1ca2f674520

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reinforcement Knowledge Graph Reasoning for Explainable Recommendation (2019, background): “At the same time, researchers have also explored RL in KG settings for other tasks such as question answering (QA) [3, 14, 29], which formulates multi-hop reasoning as a sequential decision making problem.”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, background): “DeepPath [72] Global 1 et = eq or −1 et 6= eq (et, eq − et) {r} Efficiency 1 length(p) Fully-connected network (FCN)”
Inductive Relation Prediction by Subgraph Reasoning (2019, methodology): “Thus, the average gain obtained by ensembling with GraIL is given by and the average gain obtained by pairwise ensembling among the KGE methods is given by, The average gain obtained by GraIL on WN18RR and NELL-995 are 1 .”
Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs (2019, background): “1784 technology (Das et al., 2018; Lin et al., 2018; Xiong et al., 2017), which can predict missing links for completion of KG.”
A Survey of Knowledge-enhanced Text Generation (2020, methodology): “proposed DeepPath, the first work that employed Markov decision process (MDP) and used RL based approaches to find paths in KGs [160].”
Learning Intents behind Interactions with Knowledge Graph for Recommendation (2021, background): “However, the sparse reward signals, huge action spaces, and policy gradientbased optimization make these networks hard to train and converge to a stable and satisfying solution [50, 52].”
Zero-shot Text Classification via Reinforced Self-training (2020, methodology): “Policy network pre-train is widely used by re-inforcement learning based methods to accelerate the training of RL agent (Silver et al., 2016; Xiong et al., 2017; Qin et al., 2018).”
LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs (2021, methodology): “Another line of KGQA meth-ods (Xu et al., 2016; Dong et al., 2015; Miller et al., 2016) aim to retrieve information from the KG using RL agent (Xiong et al., 2017; Qiu et al., 2020a; Lin et al., 2018; Das et al., 2018) or graph nets (Sun et al., 2018a; 2019a; Xiong et al., 2019), however, these…”
Survey on reinforcement learning for language processing (2021, background): “DeepPath [130], MINERVA[24] and M-Walk [103] are recent examples of systems that perform multi-step reasoning on a knowledge base through the use of reinforcement learning.”
Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings (2020, methodology): “Here we detail our pre-processing of the NELL995 dataset, which is originally presented by Xiong et al. (2017).”
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, neutral): “Itbuildstherewardfunctionbasedontheembeddingsgenerated bytheRGCN[74],whichmakesthosepath-basedmodelsmore explainable.”
Jointly Learning Explainable Rules for Recommendation with Knowledge Graph (2019, background): “Lots of research are focus on reasoning, such as [40, 42].”
Deep Learning on Graphs: A Survey (2018, neutral): “DeepPath [120] and MINERVA [121] both adopted RL for knowledge graph (KG) reasoning.”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “DeepPath [259] models the state space as (pre-trained) translation-based representations of entities and their induced relations.”
Multi-relational Poincaré Graph Embeddings (2019, methodology): “3), we also perform experiments on NELL-995 [36], containing 75,492 entities and 200 relations, ∼ 22% of which hierarchical.”
Knowledge Graphs: Opportunities and Challenges (2023, methodology): “The knowledge reasoning methods are mainly categorized into logic rule-based (De Meester et al. 2021), distributed representation-based (Chen et al. 2020b), and neural network-based methods (Xiong et al. 2017).”