The Winograd Schema Challenge

By Hector J. Levesque, Ernest Davis, Leora Morgenstern. 2011

1,578 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019, methodology): “WNLI Winograd NLI is a small natural language inference dataset deriving from (Levesque et al., 2011).”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, background): “WNLI The Winograd Schema Challenge (Levesque et al., 2011) is a reading comprehension challenge where each example consists of a sentence containing a pronoun and a list of its possible referents in the sentence.”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, background): “All data was sourced from TensorFlow Datasets.9 GLUE [Wang et al., 2018] and SuperGLUE [Wang et al., 2019b] each comprise a collection of text classification tasks meant to test general language understanding abilities: • Sentence acceptability judgment (CoLA [Warstadt et al., 2018]) • Sentiment ana”
RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019, methodology): “uestion NLI (QNLI) (Rajpurkar et al., 2016), Recognizing Textual Entailment (RTE) (Dagan et al. 2006; Bar-Haim et al. 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009) and Winograd NLI (WNLI) (Levesque et al., 2011). V2.0 some questions are not answered in the provided context, making the tas”
PaLM: Scaling Language Modeling with Pathways (2022, neutral): “Examples: SQuADv2, Winograd.”
Adversarial Training for Large Neural Language Models (2020, background): “…analysis (Socher et al., 2013), text similarity (Cer et al., 2017), paraphrase detection (Dolan and Brockett, 2005), and natural language inference (NLI) (Da-gan et al., 2006; Bar-Haim et al., 2006; Giampic-colo et al., 2007; Bentivogli et al., 2009; Levesque et al., 2012; Williams et al., 2018).”
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2019, methodology): “Tables 3 and 2 compares the performance of BART with several recent approaches on the well-studied SQuAD and GLUE tasks (Warstadt et al., 2018; Socher et al., 2013; Dolan & Brockett, 2005; Agirre et al., 2007; Williams et al., 2017; Dagan et al., 2006; Levesque et al., 2011).”
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (2019, background): “…(QQP; Iyer et al., 2017), Multi-Genre NLI (MNLI; Williams et al., 2018), Question NLI (QNLI; Rajpurkar et al., 2016), Recognizing Textual Entailment (RTE; Dagan et al., 2005; Bar-Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009) and Winograd NLI (WNLI; Levesque et al., 2012).”
WinoGrande (2019, methodology): “The Winograd Schema Challenge (WSC) (Levesque, Davis, and Morgenstern 2011), proposed as an alternative to the Turing Test (Turing 1950), has been used as a benchmark for evaluating commonsense reasoning.”
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, methodology): “The training and validation examples are drawn from the original WSC data (Levesque et al., 2012), as well as those distributed by the afﬁliated organization Commonsense Reasoning .”
Multi-Task Deep Neural Networks for Natural Language Understanding (2019, methodology): “WNLI The Winograd NLI (WNLI) is a natural language inference dataset derived from the Wino-grad Schema dataset (Levesque et al., 2012).”
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension (2018, background): “Commonsense Reasoning ROCStories Corpus (Mostafazadeh et al., 2016), SWAG (Zellers et al., 2018), and The Winograd Schema Challenge (WSC) (Levesque et al., 2011) are related Re Co RD : (1) ROCStories assesses commonsense reasoning in story understanding by choosing the correct story ending from…”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “We started with examples taken from the Winograd Schema Challenge (Levesque et al., 2011) and modified them to ensure that answers were a single word from”
A Simple Method for Commonsense Reasoning (2018, methodology): “This task consists of 273 questions and is designed to work against techniques such as traditional linguistic restrictions, common heuristics or simple statistical test over text corpora (" Google-proof ") [1].”
Gender Bias in Coreference Resolution (2018, methodology): “…surgeon,” but does not for the corresponding female pronoun. the style of Winograd schemas , wherein a pro-noun must be resolved to one of two previously-mentioned entities in a sentence designed to be easy for humans to interpret, but challenging for data-driven systems (Levesque et al., 2011).”
CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “Commonsense reasoning (Levesque et al., 2012) and abstractive summarization (Rush et al., 2015) represent two areas where these challenges remain readily apparent (Kry ´ sci ´ nski et al., 2019).”
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding (2019, background): “…of nine NLU tasks as in Table 1, covering textual entailment (RTE [26] and MNLI [27]), question-answer entailment (QNLI [13]), paraphrase (MRPC [28]), question paraphrase (QQP), textual similarity (STS-B [3]), sentiment (SST-2 [5]), linguistic acceptability (CoLA), and Winograd Schema (WNLI [29]).”