Data Distillation for Controlling Specificity in Dialogue Generation

By Jiwei Li, Will Monroe, Dan Jurafsky. 2017

22 citations recorded by the source corpus.

People speak at different levels of specificity in different situations. Depending on their knowledge, interlocutors, mood, etc.} A conversational agent should have this ability and know when to be specific and when to be general. We propose an approach that gives a neural network--based conversational agent this ability. Our approach involves alternating between \emph{data distillation} and model training : removing training examples that are closest to the responses most commonly produced by the model trained from the last round and then retrain the model on the remaining dataset. Dialogue generation models trained with different degrees of data distillation manifest different levels of specificity. We then train a reinforcement learning system for selecting among this pool of generation models, to choose the best level of specificity for a given input. Compared to the original generative model trained without distillation, the proposed system is capable of generating more interesting and higher-quality responses, in addition to appropriately adjusting specificity depending on the context. Our research constitutes a specific case of a broader approach involving training multiple subsystems from a single dataset distinguished by differences in a specific property one wishes to model. We show that from such a set of subsystems, one can use reinforcement learning to build a system that tailors its output to different input contexts at test time.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators (2019, methodology): “Li et al. (2017) use distillation to train different models at different levels of speciﬁcity and use reinforcement learning to pick the appropriate system response.”
Learning to Control the Specificity in Neural Response Generation (2018, methodology): “Later, Li et al. (2017) proposed a data distillation method, which trains a series of generative models at different levels of speciﬁcity and uses a reinforcement learning model to choose the model best suited for decoding depending on the conversation context.”
Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization (2021, methodology): “[20] proposed an iteration-based way to distill data, which operates as: a neural sequence-to-sequence (S2S) model is first trained and used to generate responses to inputs in a dataset.”