Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis
By Amr El-Desoky Mousa, Björn Schuller. 2017
96 citations recorded by the source corpus.
Traditional learning-based approaches to sentiment analysis of written text use the concept of bag-of-words or bag-of-n-grams, where a document is viewed as a set of terms or short combinations of terms disregarding grammar rules or word order. Novel approaches de-emphasize this concept and view the problem as a sequence classification problem. In this context, recurrent neural networks (RNNs) have achieved significant success. The idea is to use RNNs as discriminative binary classifiers to predict a positive or negative sentiment label at every word position then perform a type of pooling to get a sentence-level polarity. Here, we investigate a novel generative approach in which a separate probability distribution is estimated for every sentiment using language models (LMs) based on long short-term memory (LSTM) RNNs. We introduce a novel type of LM using a modified version of bidirectional LSTM (BLSTM) called contextual BLSTM (cBLSTM), where the probability of a word is estimated based on its full left and right contexts. Our approach is compared with a BLSTM binary classifier. Significant improvements are observed in classifying the IMDB movie review dataset. Further improvements are achieved via model combination.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Erik Cambria
- Devamanyu Hazarika
- Soujanya Poria
- Tom Young
- Navonil Majumder
- Wei Zhao
- Louis-philippe Morency
- Antonio Liotta
Citations in the corpus
- Recent Trends in Deep Learning Based Natural Language Processing (2017, methodology): “The training of the biLM model involves modeling the log-likelihood of both the sentence orientations. Finally, hidden representations from both LMs are concetenated to compose the final token vectors [42]. For each tokem, ELMo extracts the intermediate layer representations from the biLM and perform”
- Investigating Capsule Networks with Dynamic Routing for Text Classification (2018, background): “, 2017) and long shortterm memory networks (LSTMs) (Tai et al., 2015; Mousa and Schuller, 2017).”
- A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, background): “To decrease the training time and complexity based on LSTM and attention mechanisms to predict the sentiment polarity, Xue et al. [151] designed a Gated Convolutional network with Aspect Embedding (GCAE) for ACSA and ATSA, which are more efficient and simpler than the RNN-based model.”