Who did What: A Large-Scale Person-Centered Cloze Dataset

By Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, David A. McAllester. 2016

142 citations recorded by the source corpus.

We have constructed a new "Who-did-What" dataset of over 200,000 fill-in-the-gap (cloze) multiple choice reading comprehension problems constructed from the LDC English Gigaword newswire corpus. The WDW dataset has a variety of novel features. First, in contrast with the CNN and Daily Mail datasets (Hermann et al., 2015) we avoid using article summaries for question formation. Instead, each problem is formed from two independent articles --- an article given as the passage to be read and a separate article on the same events used to form the question. Second, we avoid anonymization --- each choice is a person named entity. Third, the problems have been filtered to remove a fraction that are easily solved by simple baselines, while remaining 84% solvable by humans. We report performance benchmarks of standard systems and propose the WDW dataset as a challenge task for the community.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (2017, background): “, 2016) or using cloze-style sentences instead of questions (Hermann et al., 2015; Onishi et al., 2016) (see Table 1 for more examples).”
Natural Questions: A Benchmark for Question Answering Research (2019, methodology): “A number of Cloze-style tasks have been proposed as a method of evaluating reading comprehension (Hermann et al., 2015; Hill et al., 2015; Paperno et al., 2016; Onishi et al., 2016).”
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering (2018, methodology): “These datasets have taken various forms: multiple-choice (Richardson et al., 2013), cloze-style (Hermann et al., 2015; Onishi et al., 2016; Hill et al., 2016), and span prediction (Rajpurkar et al., 2016; Trischler et al., 2017; Joshi et al., 2017) However, analysis (Chen et al., 2016; Sug-awara et…”
RACE: Large-scale ReAding Comprehension Dataset From Examinations (2017, methodology): “On RACE, Sliding Window is 28.6% better than the random baseline, while the improvement is 58.5%, 92.2% and 50% for CBTN, CBT-C and WDW.”
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine (2017, background): “The latter was followed by Hill et al. (2015), Ra-jpurkar et al. (2016) and Onishi et al. (2016), each of which has released a set of large-scale closed-world question-answer pairs focused on a speciﬁc aspect of question-answering.”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “Recently, interest in question answering (QA) from unstructured documents has increased along with the availability of large scale datasets for reading comprehension (Hermann et al., 2015; Hill et al., 2015; Rajpurkar et al., 2016; Onishi et al., 2016; Nguyen et al., 2016; Trischler et al., 2016a).”
Constructing Datasets for Multi-hop Reading Comprehension Across Documents (2017, background): “…(Yang et al., 2015b; Rajpurkar et al., 2016; Hewlett et al., 2016), web search queries (Nguyen et al., 2016), news articles (Hermann et al., 2015; Onishi et al., 2016), books (Hill et al., 2015; Paperno et al., 2016), science exams (Welbl et al., 2017), and trivia (Boyd-Graber et al., 2012; Dunn…”
The Web as a Knowledge-Base for Answering Complex Questions (2018, background): “While many QA datasets have been developed recently (Yang et al., 2015; Rajpurkar et al., 2016; Hewlett et al., 2016; Nguyen et al., 2016; Onishi et al., 2016; Hill et al., 2015; Welbl et al., 2017), they lack a focus on the importance of question decomposition.”
Gated-Attention Readers for Text Comprehension (2016, methodology): “tgz The final dataset is Who Did What4 (WDW) (Onishi et al., 2016), constructed from the LDC English Gigaword newswire corpus.”
Crowdsourcing Multiple Choice Science Questions (2017, background): “…2016; Hewlett et al., 2016) and web search user queries (Nguyen et al., 2016); for reading comprehension (RC) based on news (Hermann et al., 2015; Onishi et al., 2016), children books (Hill et al., 2015) and novels (Pa-perno et al., 2016), and for recognizing textual entailment based on image…”
Quasar: Datasets for Question Answering by Search and Reading (2017, neutral): “Children’s Book Test (CBT) (Hill et al., 2016) and Who-Did-What (WDW) (Onishi et al., 2016) are similar cloze-style datasets.”
What Makes Reading Comprehension Questions Easier? (2018, positive): “It achieved state-of-the-art-performance on the CNN/Daily Mail and Who-did-What datasets.”
DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension (2019, background): “…we have seen increased interest in large-scale cloze/span-based reading comprehension dataset construction (Hermann et al., 2015; Hill et al., 2016; Onishi et al., 2016; Rajpurkar et al., 2016; Bajgar et al., 2016; Nguyen et al., 2016; Trischler et al., 2017; Joshi et al., 2017; Choi et al., 2018).”
TVQA+: Spatio-Temporal Grounding for Video Question Answering (2019, methodology): “Given the question, q, and the answers, {ak}(5)k=1, we first formulate them as 5 hypotheses (QA-pair) hk = [q, ak] and predict their correctness scores based on the video and subtitle context, which is similar to [27, 19].”
MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension (2017, background): “Researchers also released cloze-style datasets(Hill et al. , 2015; Hermann et al. , 2015; Onishi et al. , 2016; Paperno et al. , 2016).”
Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph (2018, background): “While Question Answering (Voorhees and Tice 2000; Wang, Smith, and Mitamura 2007; Yang, Yih, and Meek 2015; Berant et al. 2013; Bordes et al. 2015; Rajpurkar et al. 2016; Nguyen et al. 2016; Onishi et al. 2016; Richardson, Burges, and Renshaw 2013; Berant et al. 2014) and Conversation Systems (Ritte”
Formal Limitations on the Measurement of Mutual Information (2018, methodology): “Related news article pairs extracted from the WhoDid-What dataset (Onishi et al., 2016)”
Embracing data abundance: BookTest Dataset for Reading Comprehension (2016, methodology): “The SQuAD dataset (Rajpurkar et al., 2016) based on Wikipedia and the Who-did-What dataset (Onishi et al., 2016) based on Giga-word news articles are factoid question-answering datasets where a multi-word answer should be extracted from a context document.”