Controlling Linguistic Style Aspects in Neural Language Generation

By Jessica Ficler, Yoav Goldberg. 2017

361 citations recorded by the source corpus.

Most work on neural natural language generation (NNLG) focus on controlling the content of the generated text. We experiment with controling several stylistic aspects of the generated text, in addition to its content. The method is based on conditioned RNN language model, where the desired content as well as the stylistic parameters serve as conditioning contexts. We demonstrate the approach on the movie reviews domain and show that it is successful in generating coherent sentences corresponding to the required linguistic style and content.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

The Curious Case of Neural Text Degeneration (2019, background): “One common approach is to shape the distribution through temperature (Goodfellow et al., 2016; Ficler and Goldberg, 2017; Fan et al., 2018).”
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “Attribute Conditioning (ATCON) Inspired by Ficler and Goldberg (2017) and Keskar et al.”
Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “…and Scaling We show self-consistency is robust to sampling strategies and parameters, by varying T in temperature sampling (Ackley et al., 1985; Ficler & Goldberg, 2017), k in top-k sampling (Fan et al., 2018; Holtzman et al., 2018; Radford et al., 2019), and p in nucleus sampling (Holtzman et…”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, methodology): “…Current methods for controlled text generation involve either ﬁne-tuning existing models with Reinforcement Learning (RL) (Ziegler et al., 2019), training Generative Ad-versarial Networks (Yu et al., 2017), or training conditional generative models (Kikuchi et al., 2016; Ficler & Goldberg, 2017).”
Rationale-Augmented Ensembles in Language Models (2022, methodology): “For sampling in the language model, we use temperature sampling (Ackley et al., 1985; Ficler & Goldberg, 2017) with temperature T = 0 .”
Survey of Hallucination in Natural Language Generation (2022, methodology): “In [50], for each training sample, the hallucination degree is estimated and converted into a categorical value as part of inputs, as in a controlled generation setting [49].”
Controllable Abstractive Summarization (2017, background): “(Ficler and Goldberg, 2017) proposes on conditional language model to generate text with stylistic requirements.”
What makes a good conversation? How controllable attributes affect human judgments (2019, background): “Controllable neural text generation Researchers have proposed several approaches to control aspects of RNN-based natural language generation such as sentiment, length, speaker style and tense (Fan et al., 2018; Ficler and Goldberg, 2017; Ghazvininejad et al., 2017; Hu et al., 2017; Kikuchi et al., 2”
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks (2018, methodology): “Perhaps the closest to the proposed method is the conditioned recurrent language model of Ficler and Goldberg (2017), which produces language with user-selected properties such as sentence length and formality but is incapable of generating para-phrases.”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, methodology): “work used custom neural models trained for their respective downstream generation tasks, including emotion-aware text generation (Ghosh et al., 2017; Ficler and Goldberg, 2017), attribute-aware product review generation (Dong et al.”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, background): “By contrast, Ficler and Goldberg (2017) extend the range of parameters used to condition the lstm, with two content-related attributes (sentiment and theme) and four stylistic parameters (length, whether the text is descriptive, whether it has a personal voice, and whether the style is professional)”
Explaining Question Answering Models through Text Generation (2020, background): “However, generation is not controlled by additional inputs (Kikuchi et al., 2016; Ficler and Goldberg, 2017; Keskar et al., 2019; Dathathri et al., 2019).”
Towards Controllable Story Generation (2018, background): “…2017; Lample et al., 2017), dialog responses (Wang et al., 2017), poems (Ghazvininejad et al., 2017), and different styles of text (Hu et al., 2017; Ficler and Goldberg, 2017; Shen et al., 2017; Fu et al., 2017). people would want to control a story generation system to produce interesting and…”
Don’t Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training (2019, background): “In terms of improving repetition and specificity, a recent alternative approach is that of control, where weighted decoding and conditional training have been explored (Fan et al., 2018; Ficler and Goldberg, 2017; Ghazvininejad et al., 2017; See et al., 2019).”
Style Transfer in Text: Exploration and Evaluation (2017, methodology): “2016), which proposed a model to embed personal information into vector representations for persona-conversation, and (Ficler and Goldberg 2017) which generated text with different contents and styles using conditional RNNs that conditioned on both content and style parameters.”
Modeling Naive Psychology of Characters in Simple Commonsense Stories (2018, background): “More generally, various projects tackle the problem of generating text from a set of attributes like sentiment or generic-ness (Ficler and Goldberg, 2017; Dong et al., 2017).”