Affect-LM: A Neural Language Model for Customizable Affective Text Generation

By Sayan Ghosh, Mathieu Chollet, Eugene Laksana, Louis-philippe Morency, Stefan Scherer. 2017

197 citations recorded by the source corpus.

Human verbal communication includes affective messages which are conveyed through use of emotionally colored words. There has been a lot of research effort in this direction but the problem of integrating state-of-the-art neural language models with affective information remains an area ripe for exploration. In this paper, we propose an extension to an LSTM (Long Short-Term Memory) language model for generation of conversational text, conditioned on affect categories. Our proposed model, Affect-LM enables us to customize the degree of emotional content in generated sentences through an additional design parameter. Perception studies conducted using Amazon Mechanical Turk show that Affect-LM can generate naturally looking emotional sentences without sacrificing grammatical correctness. Affect-LM also learns affect-discriminative word representations, and perplexity experiments show that additional affective information in conversational text can improve language model prediction.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

In Generative Models

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “Vocabulary Shifting (VOCAB-SHIFT) Inspired by Eisenstein et al. (2011) and Ghosh et al. (2017), we learn a 2-dimensional representation of toxicity and non-toxicity for every token in GPT-2’s vocabulary, which we then use to boost the likelihood of non-toxic tokens.”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, methodology): “…a backbone, most work used custom neural models trained for their respective downstream generation tasks, including emotion-aware text generation (Ghosh et al., 2017; Ficler and Goldberg, 2017), attribute-aware product review generation (Dong et al., 2017), and friendly or empathetic dialogue…”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, methodology): “research on modeling empathetic interactions in conversation [100]. We will present dialog datasets in Section7. Controlling the emotion or sentiment has become a popular topic in language generation [34, 45, 97]. In [97], an RNN-based language model is trained on large-scale review data where some ”
Modeling Naive Psychology of Characters in Simple Commonsense Stories (2018, background): “Ghosh et al. (2017) modulate language model distributions by increasing the probability of words that express certain affective LIWC (Tausczik and Pennebaker, 2016) categories.”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, methodology): “[47] or seq2seq models customized to generate emotionally-conditioned text, Zhou et al.”
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (2023, methodology): “The decoder-only structure (Liu et al., 2018; Radford et al., 2019; Dong et al., 2019; Brown et al., 2020; Ghosh et al., 2017) has many obvious advantages over the commonly used ED framework.”
Machine Learning for Synthetic Data Generation: a Review (2023, methodology): “The language model can be used to generate all types of sequence data, such as natural language [136], electronic health records [142], etc.”
Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory (2017, methodology): “Second , it is difﬁcult to consider emotions in a natural and coherent way because we need to balance grammaticality and expressions of emotions, as argued in (Ghosh et al. 2017).”
A Discriminator Improves Unconditional Text Generation without Updating the Generator (2020, unknown): “Text generation is an important part of many applications such as machine translation (Wu et al., 2016; Bahdanau et al., 2015), dialog systems (Du and Black, 2019; Ghosh et al., 2017) and image caption generation (Liu et al., 2018; Vinyals et al., 2015).”
On-the-Fly Controlled Text Generation with Experts and Anti-Experts (2021, background): “…a backbone, most work used custom neural models trained for their respective downstream generation tasks, including emotion-aware text generation (Ghosh et al., 2017; Ficler and Goldberg, 2017), attribute-aware product review generation (Dong et al., 2017), and friendly or empathetic dialogue…”
Towards Content Transfer through Grounded Text Generation (2019, background): “…a growing interest in controlling text for various form-related and linguistic properties, such as style (Ficler and Goldberg, 2017), affect (Ghosh et al., 2017), politeness (Sennrich et al., 2016), persona (Li et al., 2016b) voice (Yamagishi et al., 2016), grammatical correctness (Ji et…”
Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation (2020, background): “Text generation is an important part of many applications such as machine translation [1, 29], dialog systems [9, 11] and image caption generation [18, 28].”