Select-additive learning: Improving generalization in multimodal sentiment analysis

By Haohan Wang, Aaksha Meghawat, Louis-philippe Morency, Eric P. Xing. 2016

176 citations recorded by the source corpus.

Multimodal sentiment analysis is drawing an increasing amount of attention these days. It enables mining of opinions in video reviews which are now available aplenty on online platforms. However, multimodal sentiment analysis has only a few high-quality data sets annotated for training machine learning algorithms. These limited resources restrict the generalizability of models, where, for example, the unique characteristics of a few speakers (e.g., wearing glasses) may become a confounding factor for the sentiment classification task. In this paper, we propose a Select-Additive Learning (SAL) procedure that improves the generalizability of trained neural networks for multimodal sentiment analysis. In our experiments, we show that our SAL approach improves prediction accuracy significantly in all three modalities (verbal, acoustic, visual), as well as in their fusion. Our results show that SAL, even when trained on one dataset, achieves good generalization across two new test datasets.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Natural Language Processing, Multimodal Learning, Sentiment & Opinion Analysis, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Soujanya Poria
  2. Erik Cambria
  3. Percy Liang
  4. Louis-philippe Morency
  5. Devamanyu Hazarika
  6. Amir Zadeh
  7. Eric P. Xing
  8. Navonil Majumder

In Experimental and Cognitive Psychology

  1. Soujanya Poria
  2. Percy Liang
  3. Louis-philippe Morency
  4. Amir Zadeh
  5. Devamanyu Hazarika
  6. Erik Cambria
  7. Navonil Majumder
  8. Roger Zimmermann

In Natural Language Processing

  1. Soujanya Poria
  2. Erik Cambria
  3. Louis-philippe Morency
  4. Percy Liang
  5. Amir Zadeh
  6. Devamanyu Hazarika
  7. Eric P. Xing
  8. Pengcheng Yin

In Multimodal Learning

  1. Soujanya Poria
  2. Percy Liang
  3. Louis-philippe Morency
  4. Amir Zadeh
  5. Devamanyu Hazarika
  6. Erik Cambria
  7. Roger Zimmermann
  8. Liu Ziyin

In Sentiment & Opinion Analysis

  1. Percy Liang
  2. Soujanya Poria
  3. Amir Zadeh
  4. Louis-philippe Morency
  5. Devamanyu Hazarika
  6. Erik Cambria
  7. Ringki Das
  8. Thoudam Doren Singh

In Signal Processing

  1. Amir Zadeh
  2. Louis-philippe Morency
  3. Soujanya Poria
  4. Percy Liang
  5. Devamanyu Hazarika
  6. Erik Cambria
  7. Ringki Das
  8. Thoudam Doren Singh

Citations in the corpus