BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
By Hu Xu, Bing Liu, Lei Shu, Philip S. Yu. 2019
791 citations recorded by the source corpus.
Question-answering plays an important role in e-commerce as it allows potential customers to actively seek crucial information about products or services to help their purchase decision making. Inspired by the recent success of machine reading comprehension (MRC) on formal documents, this paper explores the potential of turning customer reviews into a large source of knowledge that can be exploited to answer user questions. We call this problem Review Reading Comprehension (RRC). To the best of our knowledge, no existing work has been done on RRC. In this work, we first build an RRC dataset called ReviewRC based on a popular benchmark for aspect-based sentiment analysis. Since ReviewRC has limited training examples for RRC (and also for aspect-based sentiment analysis), we then explore a novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC. To show the generality of the approach, the proposed post-training is also applied to some other review-based tasks such as aspect extraction and aspect sentiment classification in aspect-based sentiment analysis. Experimental results demonstrate that the proposed post-training is highly effective.
Subfields: Artificial Intelligence, Large Language Models, Question Answering & Reading Comprehension, Sentiment & Opinion Analysis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Large Language Models
In Question Answering & Reading Comprehension
In Sentiment & Opinion Analysis
Citations in the corpus
- Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “tion in (Xu et al., 2019), where the aspect term and review sentence are concatenated via [SEP].”
- CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021, methodology): “The proposed technique CLASSIC adopts the BERT-based ASC formulation in (Xu et al., 2019), where the aspect term (e.”
- Continual Pre-training of Language Models (2023, methodology): “For the three ASC tasks (see Table 1), we adopt the ASC formulation in Xu et al. (2019), where the aspect (e.g., “ sound ”) and review sentence (e.g., “ The sound is great ”) are concatenated via </s> .”
- Transformer models for text-based emotion detection: a review of BERT-based approaches (2021, methodology): “To further enhance the adaptability of the chosen language model, the authors post-trained the model by Xu et al. (2019) via MLM and NSP.”
- Domain Adaptive Training BERT for Response Selection (2019, methodology): “DL2R (Yan et al., 2016) proposed a method of re-formulating the response with utterances.”
- UDALM: Unsupervised Domain Adaptation through Language Modeling (2021, background): “This is consistent with previous works on supervised (Gururangan et al., 2020; Xu et al., 2019; Sun et al., 2019) and unsupervised settings (Han and Eisenstein, 2019; Du et al., 2020).”
- Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks (2019, methodology): “Xu et al. (2019) utilizes additional in-domain datasets to post-train BERT’s weights and then fine-tune it on this task.”
- Learning Rewards from Linguistic Feedback (2020, background): “Hybridizing sentiment and learned approaches (Jiang et al. 2011; Xu et al. 2019) could offer the best of both.”
- Natural Language Processing Advancements By Deep Learning: A Survey (2020, background): “tasks [69], post-training approaches [127] on the popular”