SemEval-2019 Task 3: EmoContext Contextual Emotion Detection in Text

By Ankush Chatterjee, Kedhar Nath Narahari, Meghana Joshi, Puneet Agrawal. 2019

332 citations recorded by the source corpus.

In this paper, we present the SemEval-2019 Task 3 - EmoContext: Contextual Emotion Detection in Text. Lack of facial expressions and voice modulations make detecting emotions in text a challenging problem. For instance, as humans, on reading “Why don’t you ever text me!” we can either interpret it as a sad or angry emotion and the same ambiguity exists for machines. However, the context of dialogue can prove helpful in detection of the emotion. In this task, given a textual dialogue i.e. an utterance along with two previous turns of context, the goal was to infer the underlying emotion of the utterance by choosing from four emotion classes - Happy, Sad, Angry and Others. To facilitate the participation in this task, textual dialogues from user interaction with a conversational agent were taken and annotated for emotion classes after several data processing steps. A training data set of 30160 dialogues, and two evaluation data sets, Test1 and Test2, containing 2755 and 5509 dialogues respectively were released to the participants. A total of 311 teams made submissions to this task. The final leader-board was evaluated on Test2 data set, and the highest ranked submission achieved 79.59 micro-averaged F1 score. Our analysis of systems submitted to the task indicate that Bi-directional LSTM was the most common choice of neural architecture used, and most of the systems had the best performance for the Sad emotion class, and the worst for the Happy emotion class.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

HappyBot: Generating Empathetic Dialogue Responses by Improving User Experience Look-ahead (2019, background): “…has been a relatively well understood and researched task (Socher et al., 2013; Felbo et al., 2017; McCann et al., 2017; Xu et al., 2018) that has been deemed necessary for generating empathetic dialogues (Fung et al., 2016; Bertero et al., 2016; Chatterjee et al., 2019a,b; Winata et al., 2019).”
Directed Acyclic Graph Network for Conversational Emotion Recognition (2021, background): “This task has been recently concerned by a considerable number of NLP researchers due to its potential applications in several areas, such as opinion mining in social media (Chatterjee et al., 2019) and building an emotional and empathetic dialog”
Sentiment Analysis: Detecting Valence, Emotions, and Other Affectual States from Text (2016, methodology): “Features from manually and automatically created sentiment lexicons such as word–valence association lexicons, word–arousal association lexicons, word–emotion association lexicons are also commonly used in conjunction with neural representations for detecting emotions or for detecting affect-related”
Transformer models for text-based emotion detection: a review of BERT-based approaches (2021, background): “The competition consisted of classifying the emotion of utterances from Twitter conversations as happy, sad, angry, or others (Chatterjee et al. 2019) given their conversational context.”
Lifelong Learning Dialogue Systems: Chatbots that Self-Learn On the Job (2020, background): “• Learning emotions and sentiments: Recognizing emotional state (Chatterjee et al. 2019) and sentiments of the user and leverage it to generate empathetic responses can be useful to building therapeutic chatbots.”
Span-based Joint Entity and Relation Extraction with Transformer Pre-training (2019, background): “The resulting models can then be transferred to target tasks with relatively small supervised training data, resulting in state-of-the-art performance in many NLP tasks such as question answering [37] or contextual emotion detection [5].”