MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation

By Jingwen Hu, Yuchen Liu, Jinming Zhao, Qin Jin. Annual Meeting of the Association for Computational Linguistics, 2021

383 citations recorded by the source corpus.

Emotion recognition in conversation (ERC) is a crucial component in affective dialogue systems, which helps the system understand users’ emotions and generate empathetic responses. However, most works focus on modeling speaker and contextual information primarily on the textual modality or simply leveraging multimodal information through feature concatenation. In order to explore a more effective way of utilizing both multimodal and long-distance contextual information, we propose a new model based on multimodal fused graph convolutional network, MMGCN, in this work. MMGCN can not only make use of multimodal dependencies effectively, but also leverage speaker information to model inter-speaker and intra-speaker dependency. We evaluate our proposed model on two public benchmark datasets, IEMOCAP and MELD, and the results prove the effectiveness of MMGCN, which outperforms other SOTA methods by a significant margin under the multimodal conversation setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/64374ad854dba86b087ed14d96a376e646103904

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Fei Zhao
  2. Chengcui Zhang
  3. Baocheng Geng

In Computer Vision and Pattern Recognition

  1. Fei Zhao
  2. Chengcui Zhang
  3. Baocheng Geng

In Experimental and Cognitive Psychology

  1. Fei Zhao
  2. Chengcui Zhang
  3. Baocheng Geng

Citations in the corpus