Thinking Like a Skeptic: Defeasible Inference in Natural Language

By Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, Yejin Choi. 2020

120 citations recorded by the source corpus.

Defeasible inference is a mode of reasoning in which an inference (X is a bird, therefore X flies) may be weakened or overturned in light of new evidence (X is a penguin). Though long recognized in classical AI and philosophy, defeasible inference has not been extensively studied in the context of contemporary data-driven research on natural language inference and commonsense reasoning. We introduce Defeasible NLI (abbreviated \delta-NLI), a dataset for defeasible inference in natural language. Defeasible NLI contains extensions to three existing inference datasets covering diverse modes of reasoning: common sense, natural language inference, and social norms. From Defeasible NLI, we develop both a classification and generation task for defeasible inference, and demonstrate that the generation task is much more challenging. Despite lagging human performance, however, generative models trained on this data are capable of writing sentences that weaken or strengthen a specified inference up to 68% of the time.

Subfields: Artificial Intelligence, Natural Language Processing, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Prompting & Reasoning

Citations in the corpus

Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, neutral): “For example, Rudinger et al. (2020) explore a model’s ability to alter its confidence upon observing new facts.”
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2 (2021, background): “…[Kassner and Schütze, 2020, Richardson and Sabharwal, 2019, Richardson et al., 2020], and can be ﬁne-tuned to correctly draw complex deductive inferences [Clark et al., 2020, Betz et al., 2020] and to generate informal reasons [Rudinger et al., 2020, Camburu et al., 2018, Brahman et al., 2020].”
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision (2020, methodology): “We focus on the Defeasible Inference task ( δ -NLI; Rudinger et al. 2020), illustrated in Figure 1.”
Think about it! Improving defeasible reasoning by first modeling the question scenario. (2021, methodology): “Datasets Our end task performance is measured on the three existing datasets for defeasible inference provided by Rudinger et al. (2020): 3 δ - ATOMIC , δ - SNLI , δ - SOCIAL (Table 1).”
Natural Language Processing with Commonsense Knowledge: A Survey (2021, background): “• Defeasible NLI [72] introduces a dataset and corresponding tasks designed to explore the concept of defeasible inference within NLP.”
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences (2020, background): “Owing to their exceptional performance across related NLG tasks (Forbes et al., 2020; Rudinger et al., 2020; Sakaguchi et al., 2020), our main interest is in evaluating pre-trained transformer language models (LMs).”