Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

By Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, Yoon Kim. North American Chapter of the Association for Computational Linguistics, 2023

353 citations recorded by the source corpus.

The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on “counterfactual” task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/f8e99be4f9a01761fab74bade2c3c18de9fc686b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Meng
2. Tianyu Gao
3. Danqi Chen
4. Cunxiang Wang
5. Jindong Wang
6. Jiatong Yu
7. Xingxu Xie
8. Hao Chen

Citations in the corpus

• What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models (2023, background): “In [25], the counterfactual reasoning ability of recent strong LLMs (GPT-4 [19], Claude [1]) is evaluated under eleven different tasks with counterfactual presuppositions.”
• Empowering Biomedical Discovery with AI Agents (2024, neutral): “As a result, model performance degrades on task variants that deviate from the assumptions made during training [175].”
• CodeMind: A Framework to Challenge Large Language Models for Code Reasoning (2024, background): “A large body of work has assessed LLMs for reasoning tasks of different modalieties (Deshpande et al., 2021; Wu et al., 2023; Miceli-Barone et al., 2023; Bubeck et al., 2023; Wang et al., 2023; Imani et al., 2023; Luo et al., 2023; Huang et al., 2023; Valmeekam et al., 2022; Min et al., 2023),…”
• A Survey on Evaluation of Large Language Models (2023, background): “[215] showed that the current LLMs have certain limitations in abstract reasoning ability.”
• Evaluating Large Language Models at Evaluating Instruction Following (2023, background): “Wu et al. (2023b) found a consistent and substantial degradation of LLMs’ performance executing the counterfactual tasks even though LLMs actually understand the instructions.”