Sentiment Analysis using Imperfect Views from Spoken Language and Acoustic Modalities

By Imran A. Sheikh, Sri Harsha Dumpala, Rupayan Chakraborty, Sunil Kumar Kopparapu. 2018

10 citations recorded by the source corpus.

Multimodal sentiment classification in practical applications may have to rely on erroneous and imperfect views, namely (a) language transcription from a speech recognizer and (b) under-performing acoustic views. This work focuses on improving the representations of these views by performing a deep canonical correlation analysis with the representations of the better performing manual transcription view. Enhanced representations of the imperfect views can be obtained even in absence of the perfect views and give an improved performance during test conditions. Evaluations on the CMU-MOSI and CMU-MOSEI datasets demonstrate the effectiveness of the proposed approach.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

Multimodal Transformer for Unaligned Multimodal Language Sequences (2019, background): “We ﬁrst evaluate MulT on the word-aligned sequences — the “home turf” of prior approaches modeling human multi-modal language (Sheikh et al., 2018; Tsai et al., 2019; Pham et al., 2019; Wang et al., 2019).”
Audio-Visual Fusion for Sentiment Classiﬁcation using Cross-Modal Autoencoder (2018, methodology): “While DCCA-based models proposed have shown improvements in the representation of ASR transcripts [23, 24], we show that the proposed crossmodal decoder based DCCA architecture gives better audio-visual representations and sentiment classification accuracies.”