SentiLARE: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis
By Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, Minlie Huang. 2019
80 citations recorded by the source corpus.
Most of the existing pre-trained language representation models neglect to consider the linguistic knowledge of texts, which can promote language understanding in NLP tasks. To benefit the downstream tasks in sentiment analysis, we propose a novel language representation model called SentiLARE, which introduces word-level linguistic knowledge including part-of-speech tag and sentiment polarity (inferred from SentiWordNet) into pre-trained models. We first propose a context-aware sentiment attention mechanism to acquire the sentiment polarity of each word with its part-of-speech tag by querying SentiWordNet. Then, we devise a new pre-training task called label-aware masked language model to construct knowledge-aware language representation. Experiments show that SentiLARE obtains new state-of-the-art performance on a variety of sentiment analysis tasks.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zhiyuan Liu
- Maosong Sun
- Zhengyan Zhang
- Yuxian Gu
- Xiaozhi Wang
- Guangyao Chen
- Guangwu Qian
- Pengcheng Gao
Citations in the corpus
- Train No Evil: Selective Masking for Task-guided Pre-training (2020, background): “For instance, in sentiment analysis, emotional tokens such as “like”, “hate” are critical for determining the sentiments of texts (Ke et al., 2019).”
- Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey (2023, background): “SentiLARE also introduces part-of-speech tag and sentiment polarity to build word-level linguistic knowledge [218].”