Recognising Textual Entailment with Logical Inference

By Johan Bos, Katja Markert. 2005

302 citations recorded by the source corpus.

We use logical inference techniques for recognising textual entailment. As the performance of theorem proving turns out to be highly dependent on not readily available background knowledge, we incorporate model building, a technique borrowed from automated reasoning, and show that it is a useful robust method to approximate entailment. Finally, we use machine learning to combine these deep semantic analysis techniques with simple shallow word overlap; the resulting hybrid model achieves high accuracy on the RTE testset, given the state of the art. Our results also show that the different techniques that we employ perform very differently on some of the subsets of the RTE corpus and as a result, it is useful to use the nature of the dataset as a feature.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “In this task, also known as recognizing textual entailment (RTE; Fyodorov et al., 2000; Condoravdi et al., 2003; Bos and Markert, 2005; Dagan et al., 2006; MacCartney and Manning, 2009), a model is presented with a pair of sentences—like one of those in Figure 1— and asked to judge the relationship…”
A Decomposable Attention Model for Natural Language Inference (2016, background): “NLI is a central problem in language understanding (Katz, 1972; Bos and Markert, 2005; Benthem, 2008; Mac-Cartney and Manning, 2009) and recently the large SNLI corpus of 570K sentence pairs was created for this task (Bowman et al., 2015).”
A large annotated corpus for learning natural language inference (2015, background): “Thus, natural language inference (NLI) — characterizing and using these relations in computational systems (Fyodorov et al., 2000; Condoravdi et al., 2003; Bos and Markert, 2005; Dagan et al., 2006; MacCartney and Manning, 2009) — is essential in tasks ranging from information retrieval to semantic ”
From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline (2024, background): “Many existing benchmarks [45, 34, 4, 11, 3, 16] are unable to keep up with the rapidly advancing capabilities of these models, becoming saturated and unable to differentiate between state-of-the-art models.”
Visual Entailment: A Novel Task for Fine-Grained Image Understanding (2019, methodology): “The text features are fused and go through two FC layers with input and output dimensions of [600, 300] and [300, 3] for the ﬁnal prediction.”
Natural Language Inference over Interaction Space (2017, methodology): “The previous work on NLI (or RTE) has extensively researched on conventional approaches(Fyodorov et al., 2000; Bos & Markert, 2005; MacCartney & Manning, 2009).”
Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment (2018, methodology): “This paper proposes a new empirical method to investigate a quality of a large corpus designed for the recognizing textual entailment (RTE) task (Condoravdi et al., 2003; Bos and Markert, 2005; MacCartney and Manning, 2009; Marelli et al., 2014a).”
An extended model of natural logic (2009, methodology): “For example, the FOL-based system of Bos and Markert (2005) was able to ﬁnd a proof for less than 4% of the problems in the RTE1 test set.”
Injecting Logical Background Knowledge into Embeddings for Relation Extraction (2015, background): “One option is to rely on a fully symbolic approach that exclusively uses first-order logic (Bos and Markert, 2005; Baader et al., 2007; Bos, 2008).”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, neutral): “We choose to focus on NLI, a canonical and wellstudied natural language understanding task (Dagan et al., 2005; Bos and Markert, 2005; Giampiccolo et al., 2007; MacCartney and Manning, 2009).”
Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention (2016, methodology): “Formal reasoning methods (Bos and Markert, 2005) were P The boy is running through a grassy area.”
EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference (2019, background): “guage understanding is natural language inference (NLI)(or recognizing textual entailment (RTE)) (Cooper et al., 1996; Condoravdi et al., 2003; Bos and Markert, 2005; Dagan et al., 2006), wherein a model determines if a natural language hypothesis”
ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment (2014, methodology): “The approaches to the task of TE can be roughly divided into two groups: (1) logic inference method (Bos and Markert, 2005) where automatic reasoning tools are used to check the logical representations derived from sentences and (2) machine learning method (Zhao et al., 2013; Gomaa and Fahmy, 2013)…”
CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank (2007, background): “…that can be obtained from the Penn Treebank are not always semantically correct (as we will discuss), subsequent work by Bos et al. (2004) and Bos (2005) has demonstrated that the output of parsers trained on CCGbank can also be directly translated into logical forms such as Discourse…”
A Natural Proof System for Natural Language (2017, methodology): “The latter dates back to Bos and Markert (2005).”
Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization (2018, background): “Generally, a correct summary is semantically entailed by the source sentence, thus we believe entailment1 (Bos and Markert, 2005) knowledge is beneficial to avoid producing contradictory or unrelated information in the summary.”
A Survey of Paraphrasing and Textual Entailment Methods (2009, methodology): “Once trained, the classifier can classify unseen pairs as correct or incorrect paraphrases or textual entailment pairs by examining their features (Bos & Markert, 2005; Brockett & Dolan, 2005; Zhang & Patrick, 2005; Finch et al., 2005; Wan et al., 2006; Burchardt et al., 2007; Hickl, 2008;…”
Learning to recognize features of valid textual entailments (2006, background): “Finally, a few efforts (Akhmatova, 2005; Fowler et al., 2005; Bos and Markert, 2005) have tried to translate sentences into formulas of ﬁrst-order logic, in order to test logical entailment with a theorem prover.”
Fact Checking: Task definition and dataset construction (2014, background): “Finally, the compilation of the answers into a verdict could be considered as a form of logic-based textual entailment (Bos and Markert, 2005).”