The Microsoft Research Sentence Completion Challenge

By Geoffry Zweig, C. Burges. 2011

83 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, background): “For instance, the negative examples could be incorrect relations in knowledge-base completion (Socher et al., 2013), or all words in a dictionary for a single-word cloze task (Zweig and Burges, 2011).”
context2vec: Learning Generic Context Embedding with Bidirectional LSTM (2016, methodology): “For the smaller 50-million-word MSCC learning corpus, a full iteration with a batch size of 100 took only about 3 hours.”
Efficient Estimation of Word Representations in Vector Space (2013, methodology): “Architecture Accuracy [%] 4-gram [29] 39 Average LSA similarity [29] 49 Log-bilinear model [22] 54.”
How Can We Know What Language Models Know? (2019, background): “…themselves can be used as a tool for text understanding by formulating queries in natural language and either generating textual answers directly (McCann et al., 2018; Radford et al., 2019), or assessing multiple choices and picking the most likely one (Zweig and Burges, 2011; Rajani et al., 2019).”
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations (2015, background): “A fundamental difference is that, where examples in the MSRCC are made of a single sentence, each query in the CBT comes with a wider context.”
Targeted Syntactic Evaluation of Language Models (2018, background): “Targeted evaluation: LM evaluation data sets using challenging prediction tasks have been proposed in the context of semantics and discourse comprehension (Zweig and Burges, 2011; Paperno et al., 2016).”
Computational Approaches to Sentence Completion (2012, methodology): “We report re-sults on a set of questions taken from a collection of SAT practice exams (Princeton-Review, 2010), and further validate the methods with the recently proposed MSR Sentence Completion Challenge set (Zweig and Burges, 2011).”
Learning Semantic Word Embeddings based on Ordinal Knowledge Constraints (2015, methodology): “The Microsoft sentence completion challenge has recently been introduced as a standard benchmark task for language modeling and other NLP techniques (Zweig and Burges, 2011).”
Regularization and nonlinearities for neural language models: when are they needed? (2013, methodology): “Humans perform on this task at around ∼ 90 % correct [13].”
Learning word embeddings efficiently with noise-contrastive estimation (2013, methodology): “MODEL CONTEXT LATENT PERCENT SIZE DIM CORRECT LSA [19] SENTENCE 300 49 SKIP-GRAM [10] 10L+10R 640 48.”