Modeling Semantic Containment and Exclusion in Natural Language Inference

By Bill MacCartney, Christopher D. Manning. 2008

225 citations recorded by the source corpus.

We propose an approach to natural language inference based on a model of natural logic, which identifies valid inferences by their lexical and syntactic features, without full semantic interpretation. We greatly extend past work in natural logic, which has focused solely on semantic containment and monotonicity, to incorporate both semantic exclusion and implicativity. Our system decomposes an inference problem into a sequence of atomic edits linking premise to hypothesis; predicts a lexical entailment relation for each edit using a statistical classifier; propagates these relations upward through a syntax tree according to semantic properties of intermediate nodes; and composes the resulting entailment relations across the edit sequence. We evaluate our system on the FraCaS test suite, and achieve a 27% reduction in error from previous work. We also show that hybridizing an existing RTE system with our natural logic system yields significant gains on the RTE3 test suite.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Improving Wikipedia verifiability with AI (2022, background): “This architecture is akin to prior work for textual entailment in natural language inference (MacCartney & Manning, 2008), i.e., testing whether a particular premise supports or contradicts a hypothesis.”
From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions (2014, background): “These operations are akin to the atomic ed-its of MacCartney and Manning (2008)’s NatLog system, and allow us to construct large subsumption hierarchies over image descriptions, which we call denotation graphs .”
Reasoning about Quantities in Natural Language (2015, background): “Monotonicity has been modeled with some success in entailment systems (Maccartney and Manning, 2008), thus providing a clear and intuitive framework for incorporating an inference resource like the Quantity Entailment module into a full textual entailment system.”
Enhanced LSTM for Natural Language Inference (2016, background): “Modeling inference in human language is notoriously challenging but is a basic problem towards true natural language understanding, as pointed out by MacCartney and Manning (2008), “a necessary (if not sufﬁcient) condition for true natural language understanding is a mastery of open-domain natural…”
An extended model of natural logic (2009, methodology): “In previous work (14), we have presented a description and evaluation of NatLog; this section summarizes the main results.”
NaturalLI: Natural Logic Inference for Common Sense Reasoning (2014, background): “Work by Reiter and McCarthy (Reiter, 1980; McCarthy, 1980) attempts to reason about the truth of a consequent in the absence of strict logical entailment. Similarly, Pearl (1989) presents a framework for assigning confidences to inferences which can be reasonably assumed.”
Diversify Your Datasets: Analyzing Generalization via Controlled Variance in Adversarial Datasets (2019, background): “, 2013), also known as natural language inference (NLI) (MacCartney and Manning, 2008; Bowman et al., 2015), a system needs to model a broad range of inference phenomena.”
Natural Logic and Natural Language Inference (2014, methodology): “Let’s now look at a more complex example (first presented in [14]) that demonstrates the interaction of a number of aspects of the model we’ve presented.”
MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity (2019, methodology): “, the system with minimal amounts of primitives and background knowledge)?; and 2) what is the lower-bound performance of such a model? Like other approaches to natural logic (MacCartney and Manning, 2008; Angeli and Manning, 2014), our model works by reasoning over surface forms (as opposed to tran”
Combining Natural Logic and Shallow Reasoning for Question Answering (2016, methodology): “We use the logic introduced by the NatLog system (MacCartney and Manning, 2007; 2008; 2009), which was in turn based on earlier theoretical work on Monotonicity Calculus (van Benthem, 1986; Sánchez Valencia, 1991).”
Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions (2010, background): “While TED permits only insert, relabel, and delete operations, edit sequences may contain more complex operations, such as moving entire subtrees and re-ordering child nodes.”
Most “babies” are “little” and most “problems” are “huge”: Compositional Entailment in Adjective-Nouns (2016, methodology): “Transformation-based RTE is widely used, not only in rule-based approaches (MacCartney and Manning, 2008; Young et al., 2014), but also in statistical RTE systems (Stern and Dagan, 2012; Pad ´ o et al., 2014).”
DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference (2018, background): “Recognizing Textual Entailment, or RTE) is an important and challenging task for natural language understanding (MacCartney and Manning, 2008).”
Exploring End-to-End Differentiable Natural Logic Modeling (2020, methodology): “For example, if NLI is deﬁned as a three-way classiﬁcation problem ( entailment , contradiction , and neutral ).”
Factuality Enhanced Language Models for Open-Ended Text Generation (2022, background): “Entailment Ratio Textual Entailment (or natural language inference) is a task of determining whether a hypothesis is entailed by, refuted by, or neutral to a given premise [65].”
DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference (2021, background): “We consider the problem of Natural Language Inference or NLI (MacCartney and Manning, 2008; Bowman et al., 2015), also known as Recognising Textual Entailment (RTE) (Dagan et al., 2005).”