Cluster-based beam search for pointer-generator chatbot grounded by knowledge

By Yik-Cheung Tam. 2020

42 citations recorded by the source corpus.

Abstract We present an end-to-end approach for knowledge-grounded response generation in Dialog System Technology Challenges 7 (DSTC7). Our system is trained by a pointer generator model, so that an output token in a response can either be generated or copied from conversation history or facts according to a trainable action probability distribution. Furthermore, to minimize generating meaningless responses, we propose using K-means to dynamically cluster and prune semantically similar partial hypotheses at each decoding step under a fixed beam budget. Moreover, we employ a language model to filter meaningless responses. Official evaluation results show that our proposed system achieved the first place in all primary automatic evaluation metrics and the overall human evaluation score.

Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Furu Wei
2. Jianfeng Gao
3. Li Dong
4. Mohit Bansal
5. Luke S. Zettlemoyer
6. M. Zhou
7. Yejin Choi
8. Wenhui Wang

In Natural Language Processing

1. Furu Wei
2. Jianfeng Gao
3. Li Dong
4. Mohit Bansal
5. M. Zhou
6. Xiaodong Liu
7. Wenhui Wang
8. Luke S. Zettlemoyer

In LLM Agents & Tool Use

1. M. Zhou
2. Nan Yang
3. Hsiao-Wuen Hon
4. Li Dong
5. Jianfeng Gao
6. Xiaodong Liu
7. Furu Wei
8. Wenhui Wang

Citations in the corpus

• Unified Language Model Pre-training for Natural Language Understanding and Generation (2019, methodology): “ to 0:5. The maximum length is 512. During decoding, we use beam search with size of 10. The maximum length of generated response is set to 40. As shown in Table 10, UNILM outperforms the best system [41] in the DSTC7 shared task [14] across all evaluation metrics. 3.5 GLUE Benchmark We evaluate UNI”
• Trading Off Diversity and Quality in Natural Language Generation (2020, background): “Vijayakumar et al. (2016), Li & Jurafsky (2016), Tam et al. (2019), and Kulikov et al. (2018) all propose modifications to beam search to force it to explore a more diverse set of beams.”
• Comparison of Diverse Decoding Methods from Conditional Language Models (2019, background): “Iterative Beam Search In an attempt to improve the size of the search space explored without sacrificing runtime, Kulikov et al. (2018) propose an iterative beam search method.”