Most “babies” are “little” and most “problems” are “huge”: Compositional Entailment in Adjective-Nouns

By Ellie Pavlick, Chris Callison-Burch. 2016

52 citations recorded by the source corpus.

We examine adjective-noun (AN) composition in the task of recognizing textual entailment (RTE). We analyze behavior of ANs in large corpora and show that, despite conventional wisdom, adjectives do not always restrict the denotation of the nouns they modify. We use natural logic to characterize the variety of entailment relations that can result from AN composition. Predicting these relations depends on context and on common-sense knowledge, making AN composition especially challenging for current RTE systems. We demonstrate the inability of current state-of-the-art systems to handle AN composition in a simpliﬁed RTE task which involves the insertion of only a single word.

Subfields: Artificial Intelligence, Natural Language Processing, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Computational Linguistics & Semantics

Citations in the corpus

Hypothesis Only Baselines in Natural Language Inference (2018, background): “Add-one RTE This mixed-genre dataset tests whether NLI systems can understand adjective-noun compounds (Pavlick and Callison-Burch, 2016).”
Evaluating Compositionality in Sentence Embeddings (2018, methodology): “Our approach is motivated partly by the need for better benchmarks to assess AI systems (White et al., 2017; Marelli et al., 2014; Pavlick & Callison-Burch, 2016; Gershman & Tenenbaum, 2015).”
Inherent Disagreements in Human Textual Inferences (2019, background): “…(Poesio and Artstein, 2005; Versley, 2008; Simons et al., 2010; Recasens et al., 2011; de Marneffe et al., 2012; Passonneau et al., 2012; Pavlick and Callison-Burch, 2016a,b; Tonhauser et al., 2018, among others) suggesting computational models would benefit from focusing on ‘‘speaker…”
Transforming Question Answering Datasets Into Natural Language Inference Datasets (2018, background): “Numerous datasets have emerged to evaluate NLI systems (Marelli et al., 2014; Pavlick and Callison-Burch, 2016; Lai et al., 2017a).”
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation (2018, background): “Recent benchmarks test whether NLI models handle adjective-noun composition (Pavlick and Callison-Burch, 2016), other types of composition (Dasgupta et al.”
Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition (2019, background): “Pavlick and Callison-Burch (2016) created a simpler variant of the recognizing textual entailment task (RTE, Dagan et al., 2013) that tests whether an adjective-noun composition entails the noun alone and vice versa in a given context.”
Towards a Unified Natural Language Inference Framework to Evaluate Sentence Representations (2018, background): “Additionally, public benchmarks exist that test whether RTE models are unable to handle adjective-noun composition (Pavlick and Callison-Burch, 2016), other types of composition (Dasgupta et al.”
Learning from others' mistakes: Avoiding dataset biases without modeling them (2020, methodology): “We evaluate on SciTail (Khot et al., 2018), GLUE benchmark’s diagnostic test (Wang et al., 2018), AddOneRTE (AddOne) Pavlick & Callison-Burch (2016), Deﬁ-nite Pronoun Resolution (DPR) (Rahman & Ng, 2012), FrameNet+ (FN+) (Pavlick et al., 2015) and Semantic Proto-Roles (SPR) (Reisinger et al., 2015).”