What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge

By Kyle Richardson, Ashish Sabharwal. 2019

50 citations recorded by the source corpus.

Abstract Open-domain question answering (QA) involves many knowledge and reasoning challenges, but are successful QA models actually learning such knowledge when trained on benchmark QA tasks? We investigate this via several new diagnostic tasks probing whether multiple-choice QA models know definitions and taxonomic reasoning—two skills widespread in existing benchmarks and fundamental to more complex reasoning. We introduce a methodology for automatically building probe datasets from expert knowledge sources, allowing for systematic control and a comprehensive evaluation. We include ways to carefully control for artifacts that may arise during this process. Our evaluation confirms that transformer-based multiple-choice QA models are already predisposed to recognize certain types of structural linguistic knowledge. However, it also reveals a more nuanced picture: their performance notably degrades even with a slight increase in the number of “hops” in the underlying taxonomic hierarchy, and with more challenging distractor candidates. Further, existing models are far from perfect when assessed at the level of clusters of semantically connected probes, such as all hypernym questions about a single concept.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

How Context Affects Language Models' Factual Predictions (2020, background): “…2018, Hewitt and Manning, 2019, Vig and Belinkov, 2019], negative polarity items [Warstadt and Bowman, 2019, Warstadt et al., 2019], semantic fragments [Richardson et al., 2019], function words [Kim et al., 2019], and many other linguistic phenomena [Tenney et al., 2019a,b, De Cao et al., 2020].”
Transformers as Soft Reasoners over Language (2020, background): “There have been numerous other demonstrations that transformers either already know [Talmor et al., 2019; Richardson and Sabharwal, 2019] or can learn to emulate other algorithms, including for semantic parsing [He and Choi, 2019], machine translation [Wang et al.”
Explaining Question Answering Models through Text Generation (2020, methodology): “Language Models (LMs), trained on large amounts of data using self-supervised learning (Peters et al., 2018; Devlin et al., 2018; Liu et al., 2019; Yang et al., 2019; Raffel et al., 2019), have been recently shown to encode substantial amounts of knowledge in their parameters (Petroni et al., 2019; ”
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2 (2021, background): “…do possess substantial conceptual knowledge, which shows in their ability to correctly draw conceptual (as opposed to formal or logical) inferences [Richardson and Sabharwal, 2019, Talmor et al., 2020a] and to rely on these relations as implicit background assumptions in answering questions…”
What Do Models Learn from Question Answering Datasets? (2020, background): “There is also growing interest in probing what models learn from datasets (McCoy et al., 2019; Geirhos et al., 2020; Richardson and Sabharwal, 2020; Si et al., 2020).”
Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation (2021, background): “Recent work also aims to probe models in these tasks to see if reasoning is actually achieved (Richardson and Sabharwal, 2020; Richardson et al., 2020; Zhou et al., 2020; Lin et al., 2021b).”
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, methodology): “…been proposed to evaluate the capabilities of handling monotonicity (Yanaka et al., 2019a), lexical inference (Glockner et al., 2018), deﬁnitions (Richardson and Sabharwal, 2019) and compositionality (Nie et al., 2019a) for RTE models and semantic equivalence (Ribeiro et al., 2019) for MRC. Liu…”
Prompting Contrastive Explanations for Commonsense Reasoning Tasks (2021, background): “ner (Petroni et al., 2020; Richardson and Sabharwal, 2020; Talmor et al., 2020; Donahue et al., 2020; Lin et al., 2019) and improving performance on downstream tasks (Brown et al.”
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms (2020, background): “Recent work also aims to probe models in these tasks to see if reasoning is actually achieved (Richardson and Sabharwal, 2020; Richardson et al., 2020).”
Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge (2020, background): “Moreover, although LM-based models do exhibit certain reasoning abilities [10, 11], these abilities are not systematic and are difﬁcult to control.”
Commonsense-Focused Dialogues for Response Generation: An Empirical Study (2021, background): “Recent work also aims to probe models in these tasks to see if reasoning is actually achieved (Richardson and Sabharwal, 2020; Richardson et al., 2020; Zhou et al., 2020).”
Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches (2021, methodology): “Synthetic test data has also been useful for better understanding what high-performing systems learn from benchmarks (Marvin and Linzen, 2018; Mc-Coy et al., 2019; Richardson and Sabharwal, 2020, inter alia ).”
Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge (2020, background): “Pre-trained LMs have been shown to capture substantial amounts of taxonomic knowledge such as hypernymy (A is a type of B) and meronymy (A is part of B) [10].”