An Attentional Neural Conversation Model with Improved Specificity

By Kaisheng Yao, Baolin Peng, Geoffry Zweig, Kam-Fai Wong. 2016

31 citations recorded by the source corpus.

In this paper we propose a neural conversation model for conducting dialogues. We demonstrate the use of this model to generate help desk responses, where users are asking questions about PC applications. Our model is distinguished by two characteristics. First, it models intention across turns with a recurrent network, and incorporates an attention model that is conditioned on the representation of intention. Secondly, it avoids generating non-specific responses by incorporating an IDF term in the objective function. The model is evaluated both as a pure generation model in which a help-desk response is generated from scratch, and as a retrieval model with performance measured using recall rates of the correct response. Experimental results indicate that the model outperforms previously proposed neural conversation architectures, and that using specificity in the objective function significantly improves performances for both generation and retrieval.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators (2019, methodology): “Yao et al. (2016) use the IDF value of generated sentences as a reward signal.”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, background): “[101] incorporated inverse document frequency (IDF) [55] into the training process to measure the response diversity.”
Learning to Control the Specificity in Neural Response Generation (2018, methodology): “Yao et al. (2016) attempted to improve the speciﬁcity with the reinforcement learning framework by using the averaged IDF score of the words in the response as a reward.”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Wu et al. (2019b), He et al. (2020a), Williams and Zweig (2016) and Yao et al. (2016) applied this pretrain-and-finetune strategy on dialogue learning and achieved outstanding performance, which proved that the reinforcement learning can improve the response quality of data-driven chatbots.”
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation (2019, methodology): “One promising line of research tackling this issue is to improve the speciﬁcity of responses, building on the intuition that generic responses frequently appear in the training data or consist of frequent words (Yao et al., 2016; Zhang et al., 2018b; Liu et al., 2018).”