Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

By Amir Zadeh, Rowan Zellers, Eli Pincus, Louis-philippe Morency. 2016

742 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Multimodal Learning, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Multimodal Learning

In Human-Computer Interaction

Citations in the corpus

Multimodal Transformer for Unaligned Multimodal Language Sequences (2019, methodology): “Meanwhile, CMU-MOSEI (Zadeh et al., 2018b) is a sentiment and emotion analysis dataset made up of 23,454 movie review video clips taken from YouTube (about 10 the size of CMU-MOSI).”
Multimodal Language Analysis with Recurrent Multistage Fusion (2018, methodology): “It is the current state of the art on CMU-MOSI and POM.”
Tensor Fusion Network for Multimodal Sentiment Analysis (2017, methodology): “Creative and novel applications of using multiple modalities have been among successful recent research directions in machine learning (You et al., 2016; Donahue et al., 2015; Antol et al., 2015; Specia et al., 2016; Tong et al., 2017). opinions from YouTube movie reviews (Zadeh et al., 2016a).”
EgoCom: A Multi-Person Multi-Modal Egocentric Communications Dataset (2020, background): “[28], and OpenFace, a multi-modal face recognition [29].”
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities (2018, positive): “The abundance of multimodal data has led to the creation of multimodal 32 datasets, such as CMU-MOSI [67] and ICT-MMMO [62], as well as deep multimodal models that are 33 highly effective at learning discriminative joint multimodal representations [30, 58, 8].”
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis (2021, methodology): “CMU-MOSEI dataset upgrades CMU-MOSI by expanding the size of the dataset.”
Multimodal sentiment analysis with word-level fusion and reinforcement learning (2017, methodology): “noisy. Multimodalsentimentanalysisintegrates verbal and nonverbal behavior to predict user sentiment. Though various multimodal datasets with sentiment labels exist [16, 21, 39], the CMU-MOSI dataset [46] is the first dataset with opinion level sentiment labels. Recent multimodal sentiment analysis ”
Context-Dependent Sentiment Analysis in User-Generated Videos (2017, methodology): “MOSI The MOSI dataset (Zadeh et al., 2016) is a dataset rich in sentimental expressions where 93 people review topics in English.”
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (2018, methodology): “We perform our experiments on the following multi-modal datasets, CMU-MOSI (Zadeh et al., 2016a POM (Park et al., 2014b), and IEMOCAP (Busso et al., 2008) for sentiment analysis, speaker traits recognition, and emotion recognition task, where the goal is to identify speakers emotions based on the…”
Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence (2019, positive): “Aside the performance of bias analysis models, results of state of the art models from MovieQA, TVQA, and CMU-MOSEI are reported in Table 1.”
Affect-LM: A Neural Language Model for Customizable Affective Text Generation (2017, background): “Multimodal Opinion-level Sentiment Intensity Dataset (CMU-MOSI): (Zadeh et al., 2016) This”
Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence Gap (2022, background): “Finally, we additionally report cross-corpus results for valence on the Multimodal Opinion Sentiment Intensity (MOSI) [49] corpus.”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, methodology): “The available datasets, such as MOSI (Zadeh et al., 2016), MOSEI (Zadeh et al.”
Learning Factorized Multimodal Representations (2018, background): “For a detailed description of the baselines, we point the reader to MFN (Zadeh et al., 2018a), MARN (Zadeh et al., 2018b), TFN (Zadeh et al., 2017), BC-LSTM (Poria et al., 2017), MV-LSTM (Rajagopalan et al., 2016), EF-LSTM (Hochreiter & Schmidhuber, 1997; Graves et al., 2013; Schuster & Paliwal, 199”
A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks (2016, methodology): “Sentiment analysis (Zadeh et al., 2016b) and using multimodal information as a new trend (Zadeh et al., 2016a; Poria et al., 2015a; Tong et al., 2017; Poria et al., 2017; Poria et al., 2016b) is a popular branch of NLP research that aims to understand sentiment of documents automatically using…”