UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification

By Aliaksei Severyn, Alessandro Moschitti. 2015

224 citations recorded by the source corpus.

This paper describes our deep learning system for sentiment analysis of tweets. The main contribution of this work is a process to initialize the parameter weights of the convolutional neural network, which is crucial to train an accurate model while avoiding the need to inject any additional features. Briefly, we use an unsupervised neural language model to initialize word embeddings that are further tuned by our deep learning model on a distant supervised corpus. At a final stage, the pre-trained parameters of the network are used to initialize the model which is then trained on the supervised training data from Semeval-2015. According to results on the official test sets, our model ranks 1st in the phrase-level subtask A (among 11 teams) and 2nd on the messagelevel subtask B (among 40 teams). Interestingly, computing an average rank over all six test sets (official and five progress test sets) puts our system 1st in both subtasks A and B.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Universal Language Model Fine-tuning for Text Classification (2018, background): “Fine-tuning Fine-tuning has been used successfully to transfer between similar tasks, e.g. in QA (Min et al., 2017), for distantly supervised sentiment analysis (Severyn and Moschitti, 2015), or MT domains (Sennrich et al., 2015) but has been shown to fail between unrelated ones (Mou et al., 2016).”
Transfer Learning in Natural Language Processing (2019, background): “, 2017) before turning to distant supervision approaches which use heuristics to automatically label datasets (Mintz et al., 2009; Severyn and Moschitti, 2015; Felbo et al., 2017; Yang et al., 2017).”
Fine-tuned Language Models for Text Classification (2018, methodology): “Fine-tuning Fine-tuning, which fine-tunes a pretrained model on the target task, has been used successfully for certain tasks such as sentiment analysis with distant supervision (Severyn and Moschitti, 2015; Felbo et al., 2017).”
Fidelity-Weighted Learning (2017, methodology): “In particular for neural networks, greedy layer-wise pre-training of weights using unlabeled data is followed by supervised ﬁne-tuning (Hinton et al., 2006; Deriu et al., 2017; Severyn & Moschitti, 2015b;a; Go et al., 2009).”