How would Stance Detection Techniques Evolve after the Launch of ChatGPT?

By Daijun Ding, Liwen Jing. 2022

148 citations recorded by the source corpus.

Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jacob Andreas
2. Ekin Akyürek
3. Jisun An
4. Fan Huang
5. Alexis Ross
6. Bailin Wang
7. Haewoon Kwak
8. Thien Huu Nguyen

In Natural Language Processing

1. Jacob Andreas
2. Ekin Akyürek
3. Jisun An
4. Fan Huang
5. Alexis Ross
6. Bailin Wang
7. Haewoon Kwak
8. Viet Dac Lai

Citations in the corpus

• Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech (2023, background): “As the most powerful text generation model, not surprisingly, researchers and practitioners have been evaluating its capability for various tasks, including question-and-answer in fnancial, medical, legal, and psychological areas [10], medical report simplifcation [13], bug fxing in computer program”
• ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identification (2023, background): “In contrast, when Zhang et al. (2022) analyzed ChatGPT’s performance on stance detection, it achieved state-ofthe-art (SOTA) results on this task.”
• Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs (2024, result): “We also point out that recent work (Aiyappa et al., 2023) showed that after model updates, Chat-GPT performance improved on benchmarks to which it was previously exposed (Zhang et al., 2022).”
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023, background): “…2023), question answering (Tan et al., 2023; Omar et al., 2023; Lai et al., 2023), information extraction (Wei et al., 2023; Gao et al., 2023), text classification (Kuz-man et al., 2023; Amin et al., 2023), grammatical error detection (Fang et al., 2023), and stance detection (Zhang et al., 2022a).”
• Zero-Shot Information Extraction via Chatting with ChatGPT (2023, methodology): “For example, (Zhang et al., 2022) use ChatGPT achieved state-of-the-art performance on Stance Detection, (Guo et al., 2023) evaluated its helpfulness on question answering, (Jiao et al., 2023) state that it is a good translator for spoken language.”
• Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, methodology): “[100] employed ChatGPT for stance detection, which includes support and opposition.”
• Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks (2023, background): “It is also possible that these are due to memorization: the model could have memorized the specific input-output mapping of a task, without understanding how to derive the output from the input, and when explicitly instructed to spell out that process, it makes more errors (Zhang et al., 2023b).”
• Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, methodology): “[106] employed ChatGPT for stance detection, which includes support and opposition.”