Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework

By Aaron Steven White, Pushpendre Rastogi, Kevin Duh, Benjamin Van Durme. 2017

89 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, methodology): “NLI has been used to investigate many other types of linguistic information besides syntactic structure (Poliak et al., 2018a; White et al., 2017).”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, neutral): “, and data from SQuAD (QNLI;2 Rajpurkar et al., 2016) and the Winograd Schema Challenge (WNLI; Levesque et al., 2011) recast as entailment in the style of White et al. (2017).”
What do you learn from context? Probing for sentence structure in contextualized word representations (2019, methodology): “In the DNC, these tasks are all recast into natural language inference (NLI) format (White et al., 2017), i.e. systems must understand the targeted semantic phenomenon in order to make correct inferences about en-9 tailment.”
Evaluating Compositionality in Sentence Embeddings (2018, methodology): “Our approach is motivated partly by the need for better benchmarks to assess AI systems (White et al., 2017; Marelli et al., 2014; Pavlick & Callison-Burch, 2016; Gershman & Tenenbaum, 2015).”
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension (2019, methodology): “The latter uses an entailment-based approach similar to White et al. (2017) and Poliak et al. (2018b), which is a ternary classiﬁcation task ( entailment , contradiction , neutral ) over sentence pairs.”
Inherent Disagreements in Human Textual Inferences (2019, methodology): “…semantics and language technology in general primarily because NLI has, historically (Cooper et al., 1996; Dagan et al., 2006) as well as presently (White et al., 2017), been proposed as a means for evaluating a model’s ‘‘intrinsic’’ understanding of language: As originally framed by Dagan et al.…”
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling (2018, methodology): “, and data from SQuAD (QNLI, Rajpurkar et al., 2016) and the Winograd Schema Challenge (WNLI, Levesque et al., 2011) recast as entailment in the style of White et al. (2017).”
Transforming Question Answering Datasets Into Natural Language Inference Datasets (2018, positive): “Although White et al. (2017) and Poliak et al. (2018) have explored recasting datasets created for various semantic classiﬁcation tasks (e.g. semantic role labeling and named entity recognition) into NLI datasets, we are the ﬁrst to perform such an automated conversion on QA datasets.”
ANLIzing the Adversarial Natural Language Inference Dataset (2020, methodology): “In comparison with White et al. (2017), which uses pre-existing semantic annotations to create an RTE/NLI formatted dataset.”
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation (2018, methodology): “paraphrastic inference, complex anaphora resolution (White et al., 2017), or compositionality (Pavlick and Callison-Burch,”
Interpretability and Analysis in Neural NLP (2020, methodology): “, 2016a; Marvin and Linzen, 2018) or inference tasks (Poliak et al., 2018a,b; White et al., 2017; Kim et al., 2019; McCoy et al., 2019; Ross and Pavlick, 2019), psycholinguistic methods (Ettinger et al.”
Lexicosyntactic Inference in Neural Models (2018, methodology): “This work is inspired by recent work in recasting various semantic annotations into natural language inference (NLI) datasets (White et al., 2017; Po-liak et al., 2018a,b; Wang et al., 2018) to gain a better understanding of which phenomena standard neural NLI models (Bowman et al., 2015; Conneau…”