MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations

By Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, Rada Mihalcea. 2018

1,647 citations recorded by the source corpus.

Emotion recognition in conversations is a challenging task that has recently gained popularity due to its potential applications. Until now, however, a large-scale multimodal multi-party emotional conversational database containing more than two speakers per dialogue was missing. Thus, we propose the Multimodal EmotionLines Dataset (MELD), an extension and enhancement of EmotionLines. MELD contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends. Each utterance is annotated with emotion and sentiment labels, and encompasses audio, visual and textual modalities. We propose several strong multimodal baselines and show the importance of contextual and multimodal information for emotion recognition in conversations. The full dataset is available for use at http://affective-meld.github.io.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Beyond Goldfish Memory: Long-Term Open-Domain Conversation (2021, background): “, 2019; Rameshkumar and Bailey, 2020) and TV shows as well (Poria et al., 2018).”
Towards Debiasing Sentence Representations (2020, background): “, 2013), a collection of 10000 polarized written movie reviews, 3) Reddit data collected from discussion forums related to politics, electronics, and relationships, 4) MELD (Poria et al., 2019), a large-scale multimodal multi-party emotional dialog dataset collected from the TV-series Friends, and 5”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Visual Dialogue; Speaker Consistency and Personality Response Shuster et al. (2018)”
CH-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality (2020, background): “In addition, Li et al. (2017) proposed a Chinese emotional audio-visual dataset and Poria et al. (2018) proposed a multi-party emotional, conversational dataset containing more than two speakers per dialogue.”
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation (2023, neutral): “Moreover, we report results on MELD under its original split setup, and RAVDESS (Liv-ingstone and Russo, 2018), SAVEE (Jackson and Haq, 2014) datasets under a random leave-one-out 10-fold CV setup, which implies at each fold, all samples within the dataset are randomly split into 80%, 10%, and 10% s”
Knowledge Aware Emotion Recognition in Textual Conversations via Multi-Task Incremental Transformer (2020, methodology): “Some of datasets, such as MELD, IEMOCAP, EmoryNLP, are multimodal conversation datasets containing textual, acoustic, and visual information.”
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation (2021, methodology): “We evaluate our proposed MMGCN model on two benchmark datasets, IEMOCAP(Busso et al., 2008) and MELD(Poria et al., 2018).”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, background): “[175] are not large enough and carry suboptimal inter-annotator agreement that impedes the performance of complex deep learning frameworks.”
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT (2023, methodology): “Considering the imbalanced data distribution of the MELD dataset, we report the weighted F1 score (WF1) as the primary metric, together with weighted accuracy (WA) and unweighted accuracy (UA).”
Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances (2019, background): “Emotional dynamics of conversations consists of two important aspects: self and inter-personal dependencies [32].”
A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, methodology): “The visual-based analysis is more effective than audio-based sentiment analysis due to audio susceptible to noise [406].”
Directed Acyclic Graph Network for Conversational Emotion Recognition (2021, background): “Recently, several ERC datasets with textual data have been released (Busso et al., 2008; Schuller et al., 2012; Zahiri and Choi, 2017; Li et al., 2017; Chen et al., 2018; Poria et al., 2019b), arousing”
Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions (2022, background): “MELD 2019 [90] A+V+T TV Series Friends Multi Speaker English Dialogues from TV seriesFriends http://affective-meld.”