Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA

By Yichen Jiang, Mohit Bansal. 2019

113 citations recorded by the source corpus.

Multi-hop question answering requires a model to connect multiple pieces of evidence scattered in a long context to answer the question. In this paper, we show that in the multi-hop HotpotQA (Yang et al., 2018) dataset, the examples often contain reasoning shortcuts through which models can directly locate the answer by word-matching the question with a sentence in the context. We demonstrate this issue by constructing adversarial documents that create contradicting answers to the shortcut but do not affect the validity of the original answer. The performance of strong baseline models drops significantly on our adversarial test, indicating that they are indeed exploiting the shortcuts rather than performing multi-hop reasoning. After adversarial training, the baseline’s performance improves but is still limited on the adversarial test. Hence, we use a control unit that dynamically attends to the question at different reasoning hops to guide the model’s multi-hop reasoning. We show that our 2-hop model trained on the regular data is more robust to the adversaries than the baseline. After adversarial training, it not only achieves significant improvements over its counterpart trained on regular data, but also outperforms the adversarially-trained baseline significantly. Finally, we sanity-check that these improvements are not obtained by exploiting potential new shortcuts in the adversarial data, but indeed due to robust multi-hop reasoning skills of the models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies (2021, background): “Moreover, answering a question based on relevant context does not imply that the model performs reasoning properly (Jiang and Bansal, 2019).”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Some of these efforts augment humans (through guidelines, templates, programmatic generation of attacks, and various combinations thereof) to devise test cases that cause systems to fail [45, 46, 29, 21, 30, 55, 6, 23].”
Break It Down: A Question Understanding Benchmark (2020, methodology): “Kwiatkowski et al. (2013) and Choi et al. (2015) used underspecified logical forms as an intermediate representation.”
Red Teaming Language Models with Language Models (2022, methodology): “Other efforts manually compose templates and code to generate test cases for specific failures (Jia and Liang, 2017; Dixon et al., 2018; Garg et al., 2019; Jiang and Bansal, 2019; Ribeiro et al., 2020).”
Unsupervised Question Decomposition for Question Answering (2020, methodology): “We report results on 3 dev set versions: (1) the original version,9 (2) the multi-hop version from Jiang and Bansal (2019a) who created some distractor paragraphs adversarially to test multi-hop reasoning, and (3) the out-of-domain (OOD) version from Min et al.”
QuALITY: Question Answering with Long Input Texts, Yes! (2021, background): “However, there has been recent work (Jiang and Bansal, 2019; Min et al., 2019) showing that these datasets contain reasoning shortcuts and a large fraction of the questions can be answered with single-hop reasoning.”
Hierarchical Graph Network for Multi-hop Question Answering (2019, background): “…the task is multi-hop by nature, recent studies (Chen and Durrett, 2019; Min et al., 2019a) also observed models that achieve high performance may not necessarily perform the expected multi-hop reasoning procedure, and may be merely leveraging some reasoning shortcuts (Jiang and Bansal, 2019a).”
Knowledge Graph Prompting for Multi-Document Question Answering (2023, background): “Existing works (Jiang and Bansal 2019) have shown that some questions in HotpotQA can be easily answered following shortcuts while questions in IIRC some-times necessitate arithmetic skills to derive answers numerically, e.g., ‘How many years did the event last when Wing-field lost his fortune?’”
Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning (2019, methodology): “We also present adversarial evaluation (Jiang and Bansal, 2019), where single-hop reasoning shortcuts are eliminated and compositional reasoning is enforced; and here our NMN again outperforms the BiDAF baseline significantly in the”
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (2022, methodology): “Adversarial HotpotQA (ADVHOTPOT) We also test on the English-language Adversarial HotpotQA dataset (Yang et al., 2018; Jiang and Bansal, 2019).”
Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention (2020, methodology): “It is a challenging combination: The retrieved documents are inevitably noisy and include much stronger distractors than the TF-IDF retrieved documents in the Distractor setting (Min et al., 2019a; Jiang & Bansal, 2019).”
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, methodology): “Adversarial Evaluation refers to generating data with the aim to “fool” a target model.”
Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge (2020, background): “, [30, 31, 32, 33, 34], where again model behavior is often opaque [35, 36, 37].”
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length (2021, methodology): “, 2020), or to perform multihop reasoning (Jiang and Bansal, 2019).”
MoreHopQA: More Than Multi-hop Reasoning (2024, background): “In the short-cuts analyses, several previous works (Min et al., 2019a; Chen and Durrett, 2019; Jiang and Bansal, 2019) also raised the issues about the multi-hop reasoning abilities of the models and the shortcuts in existing datasets.”
The Unreliability of Explanations in Few-Shot In-Context Learning (2022, methodology): “Adversarial HotpotQA (A DV H OTPOT ) We also test on the Adversarial HotpotQA dataset (Yang et al., 2018; Jiang and Bansal, 2019).”