Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos

By Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-philippe Morency, Roger Zimmermann. North American Chapter of the Association for Computational Linguistics, 2018

473 citations recorded by the source corpus.

Emotion recognition in conversations is crucial for the development of empathetic machines. Present methods mostly ignore the role of inter-speaker dependency relations while classifying emotions in conversations. In this paper, we address recognizing utterance-level emotions in dyadic conversational videos. We propose a deep neural framework, termed Conversational Memory Network (CMN), which leverages contextual information from the conversation history. In particular, CMN uses multimodal approach comprising audio, visual and textual features with gated recurrent units to model past utterances of each speaker into memories. These memories are then merged using attention-based hops to capture inter-speaker dependencies. Experiments show a significant improvement of 3 − 4% in accuracy over the state of the art.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/7fb616ff4855b7472e7054059480b9258e5e58f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation (2021, methodology): “Different models have been proposed to capture the contextual information in previous works, including the LSTM-based model (Poria et al., 2017), the conversational memory network (CMN) model (Hazarika et al., 2018b), interactive conversational memory network (ICON) model (Hazarika et al., 2018a), a”
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations (2018, methodology): “Recently, Hazarika et al. (2018) proposed a multi-modal memory network that can recognize emotion in dyadic dialogues.”
A survey on empathetic dialogue systems (2020, background): “Existing work has shown that multimodality helps to improve the accuracy of emotion detection from dialogue [143, 144, 145, 146, 147].”
Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations (2019, background): “In recent years, there is an emerging research trend on emotion detection in conversational videos and multi-turn Tweets using deep learning methods (Hazarika et al., 2018b,a; Zahiri and Choi, 2018; Chatterjee et al., 2019; Zhong and Miao, 2019; Poria et al., 2019).”
Directed Acyclic Graph Network for Conversational Emotion Recognition (2021, background): “, 2018a) and CMN (Hazarika et al., 2018b) both utilize gated recurrent unit (GRU) and memory networks.”
Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling (2018, background): “Sentiment analysis is a branch of affective computing research [3] that aims to classify text – but sometimes also audio and video [4] – into either positive or negative – but sometimes also neutral [5].”
Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances (2019, background): “[38] for dyadic dialogues, is one of the first ERC approaches that”
DialogueRNN: An Attentive RNN for Emotion Detection in Conversations (2018, methodology): “For a fair comparison with the state-of-the-art method, conversational memory networks (CMN) (Hazarika et al. 2018), we follow identical feature extraction procedures.”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, background): “Green shows primary modalities responsible for sentiment and emotion. words in the context (Tang et al., 2016a). Besides, modeling the relationships between aspects also holds potential in this task (Hazarika et al., 2018c). For example, in the sentence ”my favs here are the tacos pastor and the tos”
COSMIC: COmmonSense knowledge for eMotion Identification in Conversations (2020, methodology): “ICON (Hazarika et al., 2018b) uses two GRU networks to learn the utterance representations for dialogues between two-participants.”
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation (2019, methodology): “Memnet, CMN, and ICON are not suitable for this dataset as they exclusively work in dyadic conversations.”
Knowledge Aware Emotion Recognition in Textual Conversations via Multi-Task Incremental Transformer (2020, background): “Hazarika et al. (2018b) proposes conversational memory network (CMN) that exploits distinct memory units for each speaker to model emotional dynamics and detect emotion in a dyadic conversation.”
Integrating Multimodal Information in Large Pretrained Transformers (2020, background): “Works in this area often focus on novel multimodal neural architectures (Pham et al., 2019; Hazarika et al., 2018) and multimodal fusion approaches (Liang et al.”