Target-Dependent Sentiment Classification With BERT

By Zhengjie Gao, Ao Feng, Xinyu Song, Xi Wu. IEEE Access, 2019

365 citations recorded by the source corpus.

Research on machine assisted text analysis follows the rapid development of digital media, and sentiment analysis is among the prevalent applications. Traditional sentiment analysis methods require complex feature engineering, and embedding representations have dominated leaderboards for a long time. However, the context-independent nature limits their representative power in rich context, hurting performance in Natural Language Processing (NLP) tasks. Bidirectional Encoder Representations from Transformers (BERT), among other pre-trained language models, beats existing best results in eleven NLP tasks (including sentence-level sentiment classification) by a large margin, which makes it the new baseline of text representation. As a more challenging task, fewer applications of BERT have been observed for sentiment classification at the aspect level. We implement three target-dependent variations of the $\mathrm {BERT_{base}}$ model, with positioned output at the target terms and an optional sentence with the target built in. Experiments on three data collections show that our TD-BERT model achieves new state-of-the-art performance, in comparison to traditional feature engineering methods, embedding-based models and earlier applications of BERT. With the successful application of BERT in many NLP tasks, our experiments try to verify if its context-aware representation can achieve similar performance improvement in aspect-based sentiment analysis. Surprisingly, coupling it with complex neural networks that used to work well with embedding representations does not show much value, sometimes with performance below the vanilla BERT-FC implementation. On the other hand, incorporation of target information shows stable accuracy improvement, and the most effective way of utilizing that information is displayed through the experiment.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/8a23a2e6ece34bec461cc41da35e65bf16873752

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Donghong Ji
2. Hao Tang
3. Chenliang Li
4. Chaitanya Kulkarni
5. Mayur Wankhade
6. A. C. Rao
7. Qiji Zhou

Citations in the corpus

• Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification (2020, methodology): “, 2018), CAPSNet (Chen and Qian, 2019), Transfer-CAPS (Chen and Qian, 2019), TGBERT (Gao et al., 2019), AS-CNN (Zhang et al.”
• Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks (2021, neutral): “• TD-BERT [60] is a BERT-based model for target-dependent sentiment analysis.”
• A survey on sentiment analysis methods, applications, and challenges (2022, methodology): “A single sentence or a pair of sentences can be represented as a successive array of tokens using the task-specific BERT architecture (Gao et al. 2019).”
• ScholarBERT: Bigger is Not Always Better (2022, background): “…as BERT (Bidirectional Encoder Representations from Trans-formers) (Devlin et al., 2019) have delivered impressive results for tasks such as named entity recognition (Hakala and Pyysalo, 2019) , question answering (Qu et al., 2019; Yang et al., 2019), and sentiment classification (Gao et al., 2019).”