Attentional Encoder Network for Targeted Sentiment Classification
By Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, Yanghui Rao. 2019
319 citations recorded by the source corpus.
Targeted sentiment classification aims at determining the sentimental tendency towards specific targets. Most of the previous approaches model context and target words using recurrent neural networks such as LSTM in conjunction with attention mechanisms. However, LSTM networks are difficult to parallelize because of their sequential nature. Moreover, since full backpropagation over the sequence requires large amounts of memory, essentially every implementation of backpropagation through time is the truncated version, which brings difficulty in remembering long-term patterns. To address these issues, this paper propose an Attentional Encoder Network (AEN) for targeted sentiment classification. Contrary to previous LSTM based works, AEN eschews complex recurrent neural networks and employs attention based encoders for the modeling between context and target, which can excavate the rich introspective and interactive semantic information from the word embeddings without considering the distance between words. This paper also raise the label unreliability issue and introduce label smoothing regularization term to the loss function for encouraging the model to be less confident with the training labels. Experimental results on three benchmark datasets demonstrate that our model achieves comparable or superior performances with a lightweight model size. 1
Subfields: Artificial Intelligence, Management Science and Operations Research, Model Architectures, Sentiment & Opinion Analysis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Management Science and Operations Research
In Model Architectures
Citations in the corpus
- An Attentive Survey of Attention Models (2019, methodology): “Sentiment analysis application has also seen multiple architectures being used with attention such as memory networks [Tang et al., 2016] and Transformer [Ambartsoumian and Popowich, 2018; Song et al., 2019].”
- AttentionFGAN: Infrared and Visible Image Fusion Using Attention-Based Generative Adversarial Networks (2021, background): “Therefore, attention mechanism has now played an important role in the applications of natural language processing [38], speech recognition [39] and computer vision [37], [40], [41].”
- A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis (2019, methodology): “Third, attention based models without effectively modeling word order (e.g., MemNet and AEN) perform worst on MAMS since they lose word order information and cannot identify which part of context describes the given aspect.”
- Target-Dependent Sentiment Classification With BERT (2019, methodology): “Some existing work [4], [13], [34], [39], [44], [46] remove ‘‘conflict’’ samples from the data since the number of such instances is very small, while keeping them in the training data makes the dataset extremely unbalanced.”
- A Survey of Human-in-the-loop for Machine Learning (2021, background): “Recently numerous neural network-based approaches have been widely utilized and demonstrated their effectiveness in solving sentiment analysis tasks [99, 100, 101, 102].”