Controlling Style in Generated Dialogue

By Eric Michael Smith, Diana Gonzalez-Rico, Emily Dinan, Y-Lan Boureau. 2020

57 citations recorded by the source corpus.

Open-domain conversation models have become good at generating natural-sounding dialogue, using very large architectures with billions of trainable parameters. The vast training data required to train these architectures aggregates many different styles, tones, and qualities. Using that data to train a single model makes it difficult to use the model as a consistent conversational agent, e.g. with a stable set of persona traits and a typical style of expression. Several architectures affording control mechanisms over generation architectures have been proposed, each with different trade-offs. However, it remains unclear whether their use in dialogue is viable, and what the trade-offs look like with the most recent state-of-the-art conversational architectures. In this work, we adapt three previously proposed controllable generation architectures to open-domain dialogue generation, controlling the style of the generation to match one among about 200 possible styles. We compare their respective performance and tradeoffs, and show how they can be used to provide insights into existing conversational datasets, and generate a varied set of styled conversation replies.

Subfields: Artificial Intelligence, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Generative Models

Citations in the corpus

An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, background): “Existing works, measure representational harms in the response generated by the NLG system via automatic classifiers such as regard (Sheng et al., 2019), sentiment (Groenwold et al., 2020), style (Smith et al., 2020), and toxicity (Dhamala et al., 2021).”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “We tag each response with a binary label indicating its level of bias, and we then perform style-controlled generation on those labels so that the model can be prompted to generate responses containing lower amounts of bias (Weston et al., 2018; Smith et al., 2020a).”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “Each conversation started with a set of persona strings randomly chosen from the set of personas used in the BlendedSkillTalk dataset, which BlenderBot was fine-tuned on (Roller et al., 2020; Smith et al., 2020b).”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “For this, we take 180 input texts from the Blended Skill Talk (BST) (Smith et al., 2020a) validation set.”
Recipes for Safety in Open-domain Chatbots (2020, methodology): “This labelled data was used in Smith et al. (2020b) to train a classiﬁer that was in turned used to label the BST datasets with styles.”
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021, background): “However, these methods do not control the detailed style of the conversational system (Smith et al., 2020).”
Director: Generator-Classifiers For Supervised Language Modeling (2022, positive): “We start with a pre-trained BlenderBot 1 model Roller et al. (2020) and ﬁne-tune it on the Blended Skill Talk (BST) tasks (Smith et al., 2020b).”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, methodology): “In addition, neural LM generation is hard to control, although there are some first steps in this direction (Khalifa et al., 2021; Smith et al., 2020b).”
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems (2022, background): “A dialogue trajectory may be long and complex (Smith et al., 2020), thus here we focus on a minimal dialogue unit: a simple tuple with an opinion question for a prompt, and the chatbot’s response to that prompt.”
"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset (2022, methodology): “We use a 3B-parameter Transformer-based style classiﬁer from the controlled generation paper of Smith et al. (2020a) to measure the probability that any generated response can be assigned to any one of 217 different styles.”
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning (2021, background): “, 2020), dialogue systems (for control, coherence and safety (Welleck et al., 2018; Smith et al., 2020; Nie et al., 2020; Xu et al., 2020a)), and instruction following (Kurita & Cho, 2020).”
Linguistic calibration through metacognition: aligning dialogue agent responses with expected correctness (2020, methodology): “…conversational architectures, Smith et al. (2020b); Madotto et al. (2020) compared several methods to achieve control in conversation; here, we use the simple method of training attribute-speciﬁc con-trol tokens that was the most effective in Smith et al. (2020b) for a variety of styles.”
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2021, background): “To improve dialogue safety, numerous work researches on generation detoxifying (Dinan et al., 2019; Smith et al., 2020a; Liu et al., 2021a).”
Plug-and-Play Conversational Models (2020, background): “Concurrently to our work, Smith et al. (2020) compare the performance and tradeoffs of three existing controllable language generation methods on 200 possible styles.”