Emotion Analysis as a Regression Problem - Dimensional Models and Their Implications on Emotion Representation and Metrical Evaluation

By Sven Buechel, Udo Hahn. 2016

82 citations recorded by the source corpus.

. Emotion analysis (EA) and sentiment analysis are closely related tasks differing in the psychological phenomenon they aim to catch. We address fine-grained models for EA which treat the computation of the emotional status of narrative documents as a regression rather than a classification problem, as performed by coarse-grained approaches. We introduce Ekman’s Basic Emotions (BE) and Russell and Mehrabian’s Valence-Arousal-Dominance (VAD) model—two major schemes of emotion representation following opposing lines of psychological research, i.e., categorical and dimensional models— and discuss problems when BEs are used in a regression approach. We present the first natural language system thoroughly evaluated for fine-grained emotion analysis using the VAD scheme. Although we only employ simple BOW features, we reach correlation values up until r = . 65 with human annotations. Furthermore, we show that the prevailing evaluation methodology relying solely on Pearson’s correlation coefficient r is deficient which leads us to the introduction of a complementary error-based metric. Due to the lack of comparable (VAD-based) systems, we, finally, introduce a novel method of mapping between VAD and BE emotion representations to create a reasonable basis for comparison. This enables us to evaluate VAD output against human BE judgments and, thus, allows for a more direct comparison with existing BE-based emotion analysis systems. Even with this, admittedly, error-prone transformation step our VAD-based system achieves state-of-the-art performance in three out of six emotion categories, out-performing all existing BE-based systems but one.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Natural Language Processing, Multimodal Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sven Buechel
2. Susanna Rücker
3. Udo Hahn
4. Kai-Wei Chang
5. John Pavlopoulos
6. Anders Søgaard
7. Yada Pruksachatkun
8. Tony Sun

In Experimental and Cognitive Psychology

1. Sven Buechel
2. Udo Hahn
3. Susanna Rücker
4. Anders Søgaard
5. Alan Mislove

In Natural Language Processing

1. Sven Buechel
2. Udo Hahn
3. Susanna Rücker
4. Tony Sun
5. Rahul Gupta
6. Yada Pruksachatkun
7. Satyapriya Krishna
8. Anders Søgaard

In Multimodal Learning

1. Udo Hahn
2. Sven Buechel
3. Susanna Rücker
4. Anders Søgaard
5. Kai-Wei Chang
6. Ronghang Hu

In Evaluation & Benchmarks

1. Udo Hahn
2. Sven Buechel
3. Susanna Rücker
4. John Pavlopoulos
5. Yada Pruksachatkun
6. Satyapriya Krishna
7. Anders Søgaard
8. Iyad Rahwan

Citations in the corpus

• BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (2021, background): “Table 6 presents the proportion of texts from the Christianity group minus the proportion of texts from the Islam group that were classified into different VAD and BE5 variables.”
• Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm (2017, background): “As these two datasets do not have prior evaluations, we evaluate against a state-of-the-art approach, which is based on a valence-arousal-dominance framework (Buechel and Hahn, 2016).”
• Toxicity Detection can be Sensitive to the Conversational Context (2021, background): “, 2016), emotional analysis (Buechel and Hahn, 2016), metaphor detection (Parde and Nielsen, 2018), toxicity detection.”
• Word Emotion Induction for Multiple Languages as a Deep Multi-Task Learning Problem (2018, methodology): “As is obvious from these examples, we consider emotion prediction as a regression, not as a classification problem (see arguments discussed in Buechel and Hahn (2016)).”
• Learning and Evaluating Emotion Lexicons for 91 Languages (2020, methodology): “…mapping (ERM), an annotation projection that translates ratings from one emotion format into another, e.g., mapping VAD labels into BE5, or vice versa (Hoffmann et al., 2012; Buechel and Hahn, 2016, 2018a; Alarcão and Fonseca, 2017; Landowska, 2018; Zhou et al., 2020; Park et al., 2019).”
• EmoBank: Studying the Impact of Annotation Perspective and Representation Format on Dimensional Emotion Analysis (2017, methodology): “As far as categorical models for emotion analysis are concerned, many studies use incompatible subsets of category systems, which limits their comparability (Buechel and Hahn, 2016; Calvo and Mac Kim, 2013).”