GoEmotions: A Dataset of Fine-Grained Emotions

By Dorottya Demszky, Dana Movshovitz-Attias, J. Ko, Alan S. Cowen, Gaurav Nemade, Sujith Ravi. 2020

1,171 citations recorded by the source corpus.

Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets with a fine-grained typology, adaptable to multiple downstream tasks. We introduce GoEmotions, the largest manually annotated dataset of 58k English Reddit comments, labeled for 27 emotion categories or Neutral. We demonstrate the high quality of the annotations via Principal Preserved Component Analysis. We conduct transfer learning experiments with existing emotion benchmarks to show that our dataset generalizes well to other domains and different emotion taxonomies. Our BERT-based model achieves an average F1-score of .46 across our proposed taxonomy, leaving much room for improvement.

Subfields: Artificial Intelligence, Social Psychology, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

A Multimodal Framework for the Detection of Hateful Memes (2020, methodology): “Taking inspiration from this work, we implemented a multi-task learning variant with emotion detection as the auxiliary task using the GoEmotions dataset [4].”
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer (2021, positive): “• Sentiment analysis, including CR (Hu and Liu, 2004), GOEMOTIONS (Demszky et al., 2020), SENTIMENT140 (Go et al., 2009), SST-2, and YELP-2 (Zhang et al., 2015).”
TU Wien at TREC DL and Podcast 2021: Simple Compression for Dense Retrieval (2021, methodology): “For the Re-Rank Entertaining sub-task of the podcast segment retrieval task, we use the output of a BERT-based emotions classifier, fine-tuned on GoEmotions dataset [3], as an additional signal to re-rank the outcome of our retrieval approaches.”
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2021, methodology): “We use a subset of the GoEmotions dataset (Demszky et al., 2020) which contains Reddit posts annotated for 28 emotions, split across predefined train (|X|train = 43,410), test (|X|test = 5,427), and validation (|X|val = 5,426) subsets.”
ChatGPT: Jack of all trades, master of none (2023, methodology): “: personalized Yes Multi-label classification GoEmotions [80] / [81] 2 No 19470 1151 10 0 (0%) 1141 28 288/6 18 Unhealthy P EN Offensiveness detection No Multi-label classification Unhealthy Conv.”
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022, background): “Example of the RoT and response annotation page is in Appendix (Figure 8).”
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2024, neutral): “(Ding et al., 2021), the relation classification dataset DocRED (Yao et al., 2019), the event detection dataset MAVEN (Wang et al., 2020), and the sentiment classification dataset GoEmotions (Demszky et al., 2020).”
Long-context LLMs Struggle with Long In-context Learning (2024, neutral): “…across a variety of real-world domains such as emotion classification from text, named entity recognition, and biological function prediction, each requiring precise differentiation among vast label spaces (Zhang et al., 2017; Sileo et al., 2019; Demszky et al., 2020; Ding et al., 2021).”
Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, unknown): “[11] Its more than 3.85 million inhabitants[12] make it the European Union's most populous city, as measured by population within city limits.”