Learning to Control the Specificity in Neural Response Generation

By Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Jun Xu, Xueqi Cheng. 2018

86 citations recorded by the source corpus.

In conversation, a general response (e.g., “I don’t know”) could correspond to a large variety of input utterances. Previous generative conversational models usually employ a single model to learn the relationship between different utterance-response pairs, thus tend to favor general and trivial responses which appear frequently. To address this problem, we propose a novel controlled response generation mechanism to handle different utterance-response relationships in terms of specificity. Specifically, we introduce an explicit specificity control variable into a sequence-to-sequence model, which interacts with the usage representation of words through a Gaussian Kernel layer, to guide the model to generate responses at different specificity levels. We describe two ways to acquire distant labels for the specificity control variable in learning. Empirical studies show that our model can significantly outperform the state-of-the-art response generation models under both automatic and human evaluations.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Towards a Human-like Open-Domain Chatbot (2020, methodology): “Recent works also explore new frameworks such as adversarial learning (Li et al., 2017; Zhang et al., 2018c), variational autoencod-ing (Zhao et al., 2017; Gu et al., 2019), or both (Gao et al., 2019b) at the cost of added complexity and less scalability.”
Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators (2019, background): “Zhou et al. (2017) and Zhang et al.”
What makes a good conversation? How controllable attributes affect human judgments (2019, methodology): “Conducting experiments on the PersonaChat task (Zhang et al., 2018b), we obtain signiﬁcantly higher engaging-ness scores than the baseline by optimizing con-trol of repetition, speciﬁcity and question-asking over multiple turns.”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “ation. Manuscript submitted to ACM 10 The most notable example is CopyNet [23], which copies infrequent words from the input to the output, thus assigning higher probabilities to those rare words. In [117], Pbias is formulated as a Gaussian distribution, which assigns higher probabilities to rare wo”
Do Massively Pretrained Language Models Make Better Storytellers? (2019, background): “While there are many proposed solutions to the problem (Li and Jurafsky, 2016; Vijayakumar et al., 2018; Baheti et al., 2018; Zhang et al., 2018; See et al., 2019), it has been shown that a primary cause is likelihood-maximizing decoding algorithms such as greedy decoding and beam search (Holtzman…”
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022, unknown): “…Since speciﬁc and informative sentences are more likely to impact the quality of the whole text, we adopt normalized inverse sentence frequency (NISF) (Zhang et al., 2018) of the output sentence which can reﬂect its speciﬁcity to weight each pattern evaluator: where the inverse sentence frequency…”
Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner (2018, background): “Cheng et al. (2018) uses an additional input signal – the speciﬁcity level of a response, which is estimated by certain heuristics at training time and can be varied during evaluation.”
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation (2019, methodology): “Zhang et al. (2018b)’s model generated more complicated sentences, but has worse grammar.”
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020, background): “…contexts (Ser-ban et al., 2016, 2017; Xing et al., 2017b; Zhang et al., 2019a); to control attributes of responses (Xu et al., 2019; Zhou et al., 2017; Zhang et al., 2018a; Wang et al., 2018; See et al., 2019); and to bias responses to some speciﬁc personas (Li et al., 2016; Zhang et al., 2018b).”
Learning to Control the Fine-grained Sentiment for Story Ending Generation (2019, methodology): “A Gaussian Kernel Layer (Luong et al., 2015; Zhang et al., 2018) is used to encourage words with sentiment intensity near to target sentiment s , and P S ( y t ) is deﬁned as follow: where 2 is the variance, � S maps the sentiment embedding into a real value, the target sentiment intensity s is the…”