Commonsense Knowledge Aware Conversation Generation with Graph Attention
By Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, Xiaoyan Zhu. 2018
523 citations recorded by the source corpus.
Commonsense knowledge is vital to many natural language processing tasks. In this paper, we present a novel open-domain conversation generation model to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. Given a user post, the model retrieves relevant knowledge graphs from a knowledge base and then encodes the graphs with a static graph attention mechanism, which augments the semantic information of the post and thus supports better understanding of the post. Then, during word generation, the model attentively reads the retrieved knowledge graphs and the knowledge triples within each graph to facilitate better generation through a dynamic graph attention mechanism. This is the first attempt that uses large-scale commonsense knowledge in conversation generation. Furthermore, unlike existing models that use knowledge triples (entities) separately and independently, our model treats each knowledge graph as a whole, which encodes more structured, connected semantic information in the graphs. Experiments show that the proposed model can generate more appropriate and informative responses than state-of-the-art baselines.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Dialogue & Conversational AI, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Jianfeng Gao
- Luc Van Gool
- Bernt Schiele
- Trevor Darrell
- Xiaogang Wang
- Yi Yang
- Cordelia Schmid
In Dialogue & Conversational AI
- Bing Liu
- Minlie Huang
- Sahisnu Mazumder
- Xiaoyan Zhu
- Can Xu
- Dilek Z. Hakkani-Tür
- Behnam Hedayatnia
- Dongyan Zhao
In Knowledge Graphs & Representation
Citations in the corpus
- Wizard of Wikipedia: Knowledge-Powered Conversational agents (2018, neutral): “Zhou et al. (2018) also links Reddit to structured knowledge.”
- Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “porate commonsense knowledge for various goals, such as in conversation generation (Zhou et al., 2018), probing language models (Petroni et al.”
- Lifelong and Continual Learning Dialogue Systems: Learning during Conversation (2021, background): “…systems) have an explicit knowledge base (KB) storing real-world facts [e.g., ( Chicago , CityOf , USA )] (Eric and Manning 2017; Young et al. 2018; Zhou et al. 2018b) to support information-seeking conversations, relevant response generation and help users with product recommendations.”
- Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “Zhou et al. [130] exploited the use of large-scale commonsense knowledge for conversation generation.”
- Attention Models in Graphs (2018, methodology): “While previous work [Han et al. 2018; Zhou et al. 2018] have applied attention on knowledge graphs – which can be considered heterogeneous graphs – these models do not distinguish between the different types of links and nodes explicitly.”
- Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation (2021, background): “Grounded Dialog Generation (GDG) In the past few years, researchers are increasingly interested in grounding machine-generated dialogs on various external information (Ghazvininejad et al., 2018; Zhou et al., 2018a,b; Gopalakrishnan et al., 2019; Zheng et al., 2020; Zhou et al., 2020).”
- Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022, background): “For example, Zhou et al. (2018) enriched the context representations of the input sequence with neighbouring concepts on ConceptNet using graph attention.”
- Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation (2021, background): “Description of the annotation for these labels are in Section 3.1. neural response generation work proposed grounding on knowledge sentences that are relevant to the dialog context (Ghazvininejad et al., 2018; Yavuz et al., 2019; Zhou et al., 2018; Roller et al., 2020b).”
- Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (2021, background): “KG’s have also risen in prominence in the context of dialogue models that propose to explicitly embed symbolic knowledge representations into a neural embedding space (Liu et al., 2019b; Zhu et al., 2017; Moon et al., 2019; Zhou et al., 2018; Xu et al., 2020).”
- WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge (2020, background): “, 2019), dialogue system (Zhou et al., 2018), reading comprehension (Wang et al.”
- Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020, methodology): “On the other hand, in CMU DoG, the crowd-workers do not refer to the external knowledge as much as those workers do in Wizard when they form the responses; (2) the reinforcement step and curriculum step are useful because the reinforcement step allows the knowledge selection module to make better us”
- Knowledge-Augmented Methods for Natural Language Processing (2023, background): “Different from natural language understanding (NLU) meth-ods, NLG methods are typically under the encoder-decoder generation framework (Sutskever et al., 2014; Bahdanau et al., 2015), which poses unique challenges for leveraging knowledge into decoding the next tokens during generation.”
- Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems (2019, methodology): “Similarly, the MIT ConceptNet [116] encompasses concepts of everyday life and their relations automatically built from text data.”
- Lifelong Knowledge Learning in Rule-based Dialogue Systems (2020, background): “Many of them also use knowledge bases (KB) (e.g., Ghazvininejad et al., 2018; Le et al., 2016; Long et al., 2017; Zhou et al., 2018; Eric and Manning, 2017; Madotto et al., 2018).”
- Improving Question Answering by Commonsense-Based Pre-Training (2018, unknown): “concepts from external knowledge for each token, and get an additional vector with a solution similar to the key-value memory network. A similar idea has also been applied to conversation generation (Zhou et al. 2018), answering complex questions (Khot, Sabharwal, and Clark 2017), task-oriented dial”
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021, methodology): “For ProphetNet-Dialog-En, we utilize Reddit comments dataset (Zhou et al., 2018; Galley et al., 2019).”