OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs
By Seungwhan Moon, Pararth Shah, Anuj Kumar, Rajen Subba. 2019
346 citations recorded by the source corpus.
We study a conversational reasoning model that strategically traverses through a large-scale common fact knowledge graph (KG) to introduce engaging and contextually diverse entities and attributes. For this study, we collect a new Open-ended Dialog <-> KG parallel corpus called OpenDialKG, where each utterance from 15K human-to-human role-playing dialogs is manually annotated with ground-truth reference to corresponding entities and paths from a large-scale KG with 1M+ facts. We then propose the DialKG Walker model that learns the symbolic transitions of dialog contexts as structured traversals over KG, and predicts natural entities to introduce given previous dialog contexts via a novel domain-agnostic, attention-based graph path decoder. Automatic and human evaluations show that our model can retrieve more natural and human-like responses than the state-of-the-art baselines or rule-based models, in both in-domain and cross-domain tasks. The proposed model also generates a KG walk path for each entity retrieved, providing a natural way to explain conversational reasoning.
Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jason Aaron Edward Weston
- Mohit Bansal
- Jianfeng Gao
- Minlie Huang
- Percy Liang
- Yejin Choi
- Luke S. Zettlemoyer
- Caiming Xiong
In Natural Language Processing
- Jason Aaron Edward Weston
- Kurt Shuster
- Y-Lan Boureau
- Mohit Bansal
- Emily Dinan
- Minlie Huang
- Jianfeng Gao
- Eric Michael Smith
In LLM Agents & Tool Use
- Minlie Huang
- Jason Aaron Edward Weston
- Xiaoyan Zhu
- Maarten de Rijke
- Jianfeng Gao
- Tom Young
- Arthur Szlam
- Zhiyuan Liu
Citations in the corpus
- Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “The current approaches to using long-term knowledge are either graph representations (Moon et al., 2019a) or unstructured text retrieval (Chen et al., 2017; Dodge et al., 2016; Dinan et al., 2019b), which is then prepended onto the dialogue history and attended over, an approach advocated by the…”
- Challenges in Building Intelligent Open-domain Dialog Systems (2019, methodology): “o generate proper responses with selected knowledge even though the knowledge has never been used. Recently, there is a significant burst in constructing document or knowledge grounded dialog corpora [17, 25, 40, 86, 87, 104, 156, 191], which will be described in Section 7 in details. 4 CONSISTENCY ”
- Language Generation with Multi-hop Reasoning on Commonsense Knowledge Graph (2020, background): “…structure has been demonstrated to be an effective approach for query answering over incomplete knowledge graphs (Das et al., 2018; Chen et al., 2018; Lin et al., 2018), multi-hop question answering (Bauer et al., 2018; Cao et al., 2019; Qiu et al., 2019) 2019; Moon et al., 2019; Liu et al., 2019).”
- A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, background): “DialKG Walker [222] traverses a symbolic knowledge graph to learn contextual transition in dialogue and predicts entity responses with attentive graph path decoder.”
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023, methodology): “, 2018), OpenDialKG (Moon et al., 2019), and CNN/Daily Mail (See et al.”
- Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs (2019, background): “Moon et al. (2019) focuses on the task of entity selection, and takes advantage of positive entities that appear in the golden response.”
- Graph Retrieval-Augmented Generation: A Survey (2024, neutral): “[102],[142],[181],[168],[42],[155],[58],[60],[101],[152],[3],[136],[90],[108],[139],[179],[23],[35],[100],[4],[133],[46],[61],[17],[140],[62] Accuracy,Hits@1,EM, Recall,F1, BERTScore, GPT-4AverageRanking WebQ[5] [159],[142],[52],[168],[107],[62] CWQ[144] [102],[142],[52],[181],[155],[60],[101],[107]”
- On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022, background): “Existing dialogue systems are typically trained on corpora crowd-sourced through online platforms (Dinan et al., 2018; Gopalakrish-nan et al., 2019; Moon et al., 2019).”
- Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues? (2020, background): “To bridge the gap, researchers consider grounding open domain dialogues by external knowledge which could be retrieved either from structured knowledge bases (Zhou et al., 2018a; Moon et al., 2019; Tuan et al., 2019) or from unstructured documents (Dinan et al., 2019).”
- Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (2021, methodology): “Based on the findings of human judgement in Tab.1 and §2.1 we find hallucination errors in a dataset like OpenDialKG are often associated with entity mentions such as names of people, movies titles, locations, etc.”
- Investigating Pretrained Language Models for Graph-to-Text Generation (2020, background): “This task is important for NLP applications such as dialogue generation (Moon et al., 2019) and question answering (Duan et al., 2017).”
- Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models (2020, background): “…generated plaintext, partially or entirely independent of the actual classification or factual content but facially plausible (Xu et al., 2015b; Liu et al., 2019); or direct exposure of structure in the underlying model, such as traversals through a graph (Yang et al., 2018; Moon et al., 2019).”
- Relational Memory-Augmented Language Models (2022, background): “, 2015), dialogue systems (Zhang et al., 2018; Moon et al., 2019; Guo et al., 2018; Liu et al., 2021b), and recommendation systems (Zhang et al.”
- A Survey of Knowledge-enhanced Text Generation (2020, background): “asoning over a KG, infer potential answer-related entities, and generate the corresponding answer. So, the challenge lies in identifying a subset of desired entities and mentioning them in a response [101]. These connected entities usually follow natural conceptual threads which are able to help gen”
- Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Conversation Topic; Dialogue Reasoning Moon et al. (2019)”
- Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, methodology): “Finally, in OpenDialKG we use the 2-hop neighbors of the entity appearing in the user turn as the gold-reference for the F1-score, which are defined as N r 2 ( n ) ∀ n ∈ E ( U t ) , ∃ r ∈ R , where E ( U t ) are the list of entity nodes appearing in U t .”
- CRSLab: An Open-Source Toolkit for Building Conversational Recommender System (2021, background): “e Action Prediction DB CNet DuRecDial (Liu et al., 2020) 10,200 156,000 Movie, Music Goal Planning CN-DB HNet INSPIRED (Hayati et al., 2020) 1,001 35,811 Movie Strategy Prediction DB CNet OpenDialKG (Moon et al., 2019) 13,802 91,209 Movie, Book Path Generation DB CNet Table 1: The collected datasets”