Overview of the NTCIR-11 QA-Lab Task

By Hideyuki Shibuki, Kotaro Sakamoto, Yoshinobu Kano, Teruko Mitamura, Madoka Ishioroshi, Kelly Y. Itakura, Di Wang, Tatsunori Mori, Noriko Kando. 2014

51 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

RACE: Large-scale ReAding Comprehension Dataset From Examinations (2017, methodology): “…et al., 2013) (containing 500 passages and 2000 questions) and several others (Peñas et al., 2014; Rodrigo et al., 2015; Khashabi et al., 2016; Shibuki et al., 2014), the usefulness of those datasets is significantly restricted due to their small sizes, especially not suitable for training…”
Improving Machine Reading Comprehension with General Reading Strategies (2018, methodology): “Besides crowdsourcing, datasets such as RACE (Lai et al., 2017) and ARC (Clark et al., 2018) are collected from language or science exams designed by educational experts (Penas et al., 2014; Shibuki et al., 2014; Tseng et al., 2016) to evaluate the comprehension level of human participants.”
Automated Historical Fact-Checking by Passage Retrieval, Word Statistics, and Virtual Question-Answering (2017, methodology): “Finally, Table 7 presents the accuracy of our system in comparison with previously reported re-sults by fully automatic systems (Shibuki et al., 2014, 2016).”
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning (2020, methodology): “Besides, NTCIR QA Lab (Shibuki et al., 2014) offers comparative evaluation for solving real-world university entrance exam questions; The dataset of CLEF QA Entrance Exams Task (Rodrigo et al., 2015) is extracted from standardized English examinations for university admission in Japan; ARC dataset (”