YouTube Movie Reviews: Sentiment Analysis in an Audio-Visual Context

By Martin Wöllmer, Felix Weninger, T. Knaup, Björn Schuller, Congkai Sun, Kenji Sagae, Louis-philippe Morency. IEEE Intelligent Systems, 2013

414 citations recorded by the source corpus.

This work focuses on automatically analyzing a speaker's sentiment in online videos containing movie reviews. In addition to textual information, this approach considers adding audio features as typically used in speech-based emotion recognition as well as video features encoding valuable valence information conveyed by the speaker. Experimental results indicate that training on written movie reviews is a promising alternative to exclusively using (spoken) in-domain data for building a system that analyzes spoken movie review videos, and that language-independent audio-visual analysis can compete with linguistic analysis.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/be32d87c03d8c49a2b48f88b9f3f17340e7c8e6f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis (2021, methodology): “We also compare our model with more recent and competitive baselines, in-models cluding BERT-based model—MAG-BERT (Rah-man et al., 2020) and Self-MM (Yu et al., 2021), which works with multi-task learning and is the SOTA method.”
Select-additive learning: Improving generalization in multimodal sentiment analysis (2016, background): “Multimodal sentiment analysis extends traditional textual sentiment analysis by analyzing all three modalities present in online videos, including acoustic and visual modalities [3, 4].”
A Review and Meta-Analysis of Multimodal Affect Detection Systems (2015, background): “…was considerable variation in model-level fusion methods, but bidirectional long short-term memories [Eyben et al. 2010; Metallinou et al. 2012; W¨ollmer et al. 2010, 2013a], various HMM-based approaches (error-weighted semicoupled HMMs [Lin et al. (e.g., Jiang et al. [2011], Paleari et al.…”
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations (2018, background): “Although there is significant work carried out on multimodal emotion recognition (Poria et al., 2017a; Zadeh et al., 2016a; Wollmer et al., 2013)”
Multimodal sentiment analysis with word-level fusion and reinforcement learning (2017, background): “Though various multimodal datasets with sentiment labels exist [16, 21, 39], the CMU-MOSI dataset [46] is the first dataset with opinion level sentiment labels.”
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (2018, background): “, 1998), (Wöllmer et al., 2013), sentiment analysis, (Morency et al.”
Memory Fusion Network for Multi-view Sequential Learning (2018, methodology): “The ICT-MMMO dataset (W¨ollmer et al. 2013) consists of online social review videos that encompass a strong diversity in how people express opinions, annotated at the video level for sentiment.”
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities (2018, neutral): “State-of-the-art results are also achieved on ICT-MMMO and YouTube (Table 2).”
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (2018, neutral): “The ICT-MMMO (W¨ollmer et al., 2013) consists of online social review videos annotated at the video level for sentiment.”
Multi-attention Recurrent Network for Human Communication Comprehension (2018, methodology): “ICT-MMMO The ICT-MMMO dataset (Wöllmer et al. 2013) consists of online social review videos that encompass a strong diversity in how people express opinions, annotated at the video level for sentiment.”
Tensor Fusion Network for Multimodal Sentiment Analysis (2017, methodology): “There exist several multimodal datasets that include sentiment annotations, including the newly-introduced CMU-MOSI dataset (Zadeh et al., 2016b), as well as other datasets including ICT-MMMO (W¨ollmer et al., 2013), YouTube (Morency et al., 2011), and MOUD (P´erez-Rosas et al., 2013), however CMU-M”
Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges (2023, methodology): “[138] carried out a multimodal sentiment analysis on online user-generated movie review videos.”