A Controllable Model of Grounded Response Generation

By Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf and 1 more. 2020

89 citations recorded by the source corpus.

Current end-to-end neural conversation models inherently lack the flexibility to impose semantic control in the response generation process, often resulting in uninteresting responses. Attempts to boost informativeness alone come at the expense of factual accuracy, as attested by pretrained language models' propensity to "hallucinate" facts. While this may be mitigated by access to background knowledge, there is scant guarantee of relevance and informativeness in generated responses. We propose a framework that we call controllable grounded response generation (CGRG), in which lexical control phrases are either provided by a user or automatically extracted by a control phrase predictor from dialogue context and grounding knowledge. Quantitative and qualitative results show that, using this framework, a transformer based model with a novel inductive attention mechanism, trained on a conversation-like Reddit dataset, outperforms strong generation baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Andrea Madotto
3. Pascale Fung
4. Erik Cambria
5. Qun Liu
6. Percy Liang
7. Zhaojiang Lin
8. Tom Young

In Computer Vision and Pattern Recognition

1. Jianfeng Gao
2. Chunyuan Li
3. Qun Liu
4. Pascale Fung
5. Andrea Madotto
6. Luc Van Gool
7. Dacheng Tao
8. Wenliang Dai

In Natural Language Processing

1. Andrea Madotto
2. Jianfeng Gao
3. Pascale Fung
4. Hua Wu
5. Haifeng Wang
6. Qun Liu
7. Zhaojiang Lin
8. Erik Cambria

In Generative Models

1. Wei Ping
2. Dan Su
3. Jianfeng Gao
4. Bryan Catanzaro
5. Chunyuan Li
6. Zhaojiang Lin
7. Haifeng Wang
8. Shi Feng

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, background): “Controllable generation techniques such as controlled re-sampling [152], control codes that can be provided manually [50, 152, 210], or predicted automatically [210] are leveraged to improve faithfulness.”
• A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, unknown): “…Input:dialoguecontent,additionalstructural information(e.g.,persona,emotion,intent,template,etc.) Output:Dialogueresponse Persona[126,127,157,167,168] politeness[36,92],Sentiment[35,113,129,145], Template[55,139], ground-truthreference[97,105,109,147] Storytelling Input:Storyelements”
• Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022, methodology): “…data in KILT benchmark (Petroni et al., 2021), including TrivaQA (Joshi et al., 2017), kwiatkowski2019naturaluestion (Kwiatkowski et al., 2019), HotpotQA (Yang et al., 2018), Fever (Thorne et al., 2018), zsRE (Levy et al., 2017), AY2, T-REx (Elsahar et al., 2018b) and WoW (Dinan et al., 2018).”
• Factuality Enhanced Language Models for Open-Ended Text Generation (2022, background): “[57] Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al.”
• SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model (2020, methodology): “For chit-chat dialog systems, DialoGPT (Zhang et al., 2019b; Wolf et al., 2019b) and CGRG (Wu et al., 2020b) extended GPT-2 (Radford et al., 2019) to ground on dialog response generation settings.”
• Plug-and-Play Conversational Models (2020, background): “, 2019), knowledge grounding (Dinan et al., 2018; Gopalakrishnan et al., 2019; Ghazvininejad et al., 2018; Moghe et al., 2018; Wu et al., 2020), emotions (Li et al.”
• Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “More recently, Wu et al. (2020a) pretrained Task-Oriented Dialogue BERT (TOD-BERT) and significantly improved the accuracy in the intent detection sub-task.”
• ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, background): “…works explored many approaches to promote several aspects of credible generation: Plan-and-write [35] uses a hierarchical method that first plans a storyline and then generates a story based on it to improve the text coherency; CGRG [36] uses grounding knowledge to generate veritable answers.”