Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets

By Jeremy Barnes, Roman Klinger, Sabine Schulte im Walde. 2017

75 citations recorded by the source corpus.

There has been a good amount of progress in sentiment analysis over the past 10 years, including the proposal of new methods and the creation of benchmark datasets. In some papers, however, there is a tendency to compare models only on one or two datasets, either because of time restraints or because the model is tailored to a specific task. Accordingly, it is hard to understand how well a certain model generalizes across different tasks and datasets. In this paper, we contribute to this situation by comparing several models on six different benchmarks, which belong to different domains and additionally have different levels of granularity (binary, 3-class, 4-class and 5-class). We show that Bi-LSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are particularly good at fine-grained sentiment tasks (i.e., with more than two classes). Incorporating sentiment information into word embeddings during training gives good results for datasets that are lexically similar to the training data. With our experiments, we contribute to a better understanding of the performance of different model architectures on different data sets. Consequently, we detect novel state-of-the-art results on the SenTube datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Transfer Learning with Neural AutoML (2018, methodology): “[41] compare many algorithms and report best accuracy on Sentiment-SST of 83.”
Fine-tune BERT with Sparse Self-Attention Mechanism (2019, methodology): “In particular, for sentiment analysis, we follow (Ambartsoumian and Popowich, 2018) to make comparisons with the following representa-1 tive methods: Ave, LSTM, BiLSTM, and CNN from (Barnes et al., 2017); SSAN from (Am-bartsoumian and Popowich, 2018).”
Artificial intelligence and illusions of understanding in scientific research (2024, unknown): “This approach is common in computational social science, in which Quants are used to estimate proxies for concepts that are not directly observable 132,133 , such as people’s political ideology 134 or emotional expressions 135,136 .”
Crowdsourcing Subjective Tasks: The Case Study of Understanding Toxicity in Online Discussions (2019, background): “As [47] points out, much of this work builds on existing machine learning approaches in fields like sentiment analysis [5, 35, 50, 52] and spam detection [37, 39].”