Abductive Commonsense Reasoning

By Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Scott Yih, Yejin Choi. 2019

542 citations recorded by the source corpus.

Abductive reasoning is inference to the most plausible explanation. For example, if Jenny finds her house in a mess when she returns from work, and remembers that she left a window open, she can hypothesize that a thief broke into her house and caused the mess, as the most plausible explanation. While abduction has long been considered to be at the core of how people interpret and read between the lines in natural language (Hobbs et al., 1988), there has been relatively little research in support of abductive natural language inference and generation. We present the first study that investigates the viability of language-based abductive reasoning. We introduce a challenge dataset, ART, that consists of over 20k commonsense narrative contexts and 200k explanations. Based on this dataset, we conceptualize two new tasks -- (i) Abductive NLI: a multiple-choice question answering task for choosing the more likely explanation, and (ii) Abductive NLG: a conditional generation task for explaining given observations in natural language. On Abductive NLI, the best model achieves 68.9% accuracy, well below human performance of 91.4%. On Abductive NLG, the current best language generators struggle even more, as they lack reasoning capabilities that are trivial for humans. Our analysis leads to new insights into the types of reasoning that deep pre-trained language models fail to perform--despite their strong performance on the related but more narrowly defined task of entailment NLI--pointing to interesting avenues for future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “Our dataset is not to be confused with abductive NLI (Bhagavatula et al., 2019), which calls itself α NLI, or ART.”
Reasoning Like Program Executors (2022, background): “Our work concentrates on improving the above reasoning skills, leaving the other reasoning abilities such as commonsense reasoning (Zellers et al., 2018; Talmor et al., 2019; Bhagavatula et al., 2020) for future work.”
UL2: Unifying Language Learning Paradigms (2022, methodology): “We use HellaSwag (Zellers et al., 2019), SocialIQA/SIQA (Sap et al., 2019), PhysicalIQA/PIQA (Bisk et al., 2020), CosmosQA (Huang et al., 2019), AbductiveNLI (Bhagavatula et al., 2019), CommonsenseQA (Talmor et al., 2018), CommonsenseQA2 (Talmor et al., 2021).”
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark (2021, methodology): “To keep evaluation clean-cut, we only chose multiple-choice question-answering datasets. α NLI (Bhagavatula et al. 2020) tests abductive reasoning in narratives.”
Adversarial Filters of Dataset Biases (2020, methodology): “Variants of this AF approach have recently been used to create other datasets such as HellaSwag (Zellers et al., 2019) and ANLI (Bhagavatula et al., 2019) by iteratively perturbing dataset instances until a target model cannot ﬁt the resulting dataset.”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, background): “Namely, language is compositional; hence, the model can dynamically manipulate the semantic concepts which can be recombined in novel situations (Lake et al., 2015) and later supported by language-based abductive reasoning (e.g. Bhagavatula et al., 2020).”
GraDA: Graph Generative Data Augmentation for Commonsense Reasoning (2022, background): “…the emergence of several datasets for improving commonsense reasoning of language models through tasks like question answering (QA) (Sap et al., 2019b; Talmor et al., 2019; Bisk et al., 2020) and natural language inference (Bhagavatula et al., 2020; Zellers et al., 2019; Sakaguchi et al., 2020).”
Diffusion-LM Improves Controllable Text Generation (2022, background): “Given a left context (O1) and a right context (O2) from the aNLG dataset [2], and the goal is to generate a sentence that logically connects O1 and O2 (algorithm details in Appendix G).”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, background): “First adoptions of NLEs have been in image classification [23] and were further extended to self-driving cars [26], visual question-answering [37], and natural language processing [11; 39; 9; 6; 27; 35; 28; 12].”
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning (2020, background): “Some evaluation sets target particular well-deﬁned aspects of commonsense, such as the physical aspect (Bisk et al., 2020), abductive reasoning (Bhagavatula et al., 2020), 12 in-tents and reactions to events (Rashkin et al., 2018), social interactions (Sap et al., 2019), or visual commonsense…”
Explanations for CommonsenseQA: New Dataset and Models (2021, background): “Bhagavatula et al. (2019) have proposed an NLI dataset for abductive reasoning.”
Language Generation with Multi-hop Reasoning on Commonsense Knowledge Graph (2020, methodology): “Second, it ignores the abundant structural relational relevance of the concepts in the knowledge graph (Guan et al., 2020; Bhagavatula et al., 2020) that may provide multiple plausible evidence for complex reasoning.”