Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews

By Grégoire Mesnil, Tomas Mikolov, Marc'Aurelio Ranzato, Yoshua Bengio. 2014

145 citations recorded by the source corpus.

Sentiment analysis is a common task in natural language processing that aims to detect polarity of a text document (typically a consumer review). In the simplest settings, we discriminate only between positive and negative sentiment, turning the task into a standard binary classification problem. We compare several ma- chine learning approaches to this problem, and combine them to achieve the best possible results. We show how to use for this task the standard generative lan- guage models, which are slightly complementary to the state of the art techniques. We achieve strong results on a well-known dataset of IMDB movie reviews. Our results are easily reproducible, as we publish also the code needed to repeat the experiments. This should simplify further advance of the state of the art, as other researchers can combine their techniques with ours with little effort.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Mohit Bansal
3. Byron C. Wallace
4. Jianfeng Gao
5. Jian Tang
6. Ye Zhang
7. Alec Radford
8. Furu Wei

Citations in the corpus

• PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks (2015, background): “For the results of PVDBOW on the imdb data set, our results are different from those reported in [10, 16].”
• Quasi-Recurrent Neural Networks (2016, neutral): “3 Ensemble of RNNs and NB-SVM (Mesnil et al., 2014) − 92.”
• Representation Stability as a Regularizer for Improved Text Analytics Transfer Learning (2017, neutral): “An ensemble over multiple diverse models trained for the same sentiment analysis task was also considered in (Mesnil et al., 2014) for the IMDB binary movie reviews sentiment dataset (Maas et al.”
• Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding (2015, unknown): “data Ensemble of 3 models [18] 7.”
• A Deep Learning and Knowledge Transfer Based Architecture for Social Media User Characteristic Determination (2015, methodology): “As we began exploring other techniques which we will describe in more detail in subsequent sections, we use Paragraph Vector as provided by the original developers (Mesnil et al., 2015).”
• Improved Deep Learning Baselines for Ubuntu Corpus Dialogs (2015, background): “We found that averaging predictions of multiple models further improves performance, which is common in many machine learning tasks [13, 14].”
• Learning to Generate Reviews and Discovering Sentiment (2017, background): “Fitting a threshold to this single unit achieves a test accuracy of 92.30% which outperforms a strong supervised results on the dataset, the 91.87% of NB-SVM trigram (Mesnil et al., 2014), but is still below the semi-supervised state of the art of 94.09% (Miyato et al., 2016).”
• Semi-Supervised Learning with Multi-View Embedding: Theory and Application with Convolutional Neural Networks (2015, unknown): “43 Ensemble+ + NB-LM [17] Unlabeled data mvCNNo&w [Ours] 6.”
• Weighted Neural Bag-of-n-grams Model: New Baselines for Text Classification (2016, methodology): “(Mesnil et al., 2014) 91.87 Paragraph Vector(Mesnil et al., 2014) 88.73 DAN(Iyyer et al., 2015) 89.40 DV-tri(Li et al., 2015a) 92.14 our model 92.95 our ensemble 93.51 sequential word2vec-LSTM(Dai and Le, 2015) 90.00 SA-LSTM(Dai and Le, 2015) 92.76 seq2-bown-CNN(Johnson and Zhang, 2015a) 92.33…”
• Sentiment Classification Using Document Embeddings Trained with Cosine Similarity (2019, unknown): “87 (Mesnil et al., 2015) DV-ngram 92.”
• Self-Taught Convolutional Neural Networks for Short Text Clustering (2017, methodology): “In our experiments, we use the open source software 11 released by Mesnil et al. [44].”
• Do Characters Abuse More Than Words? (2016, methodology): “In this way, if the probability of a comment given the abusive language model is higher than its probability given the non-abusive language model, then the comment is classified as abusive and vice versa (Mesnil et al., 2014) and their ratio is used to calculate the AUC metric.”
• Neural information retrieval: at the end of the early years (2017, background): “Perhaps most significantly, later work co-authored by Mesnil et al. (2014)9 has disavowed the original findings of Le and Mikolov (2014), writing, ‘‘to match the results from Le and Mikolov (2014), we followed the [author’s] suggestion ...”
• A survey on classification techniques for opinion mining and sentiment analysis (2017, methodology): “…condition y(i) = +1} p−(x|y=+1) Means {x(i)with condition y(i) = −1} According to Bayes rule at the test time, for input (x) we have relation 9 (Mesnil et al. 2015): R = p + (x|y = +1) p− (x|y = −1) × p (y = 1) p (y = −1) (9) In relation 9, if R > 1, x belongs to the positive class,…”
• A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs (2016, unknown): “2 NBSVM-tri, RNN, Sentence-Vec Ensemble (Mesnil et al., 2014) − − 92.”