A study of the knowledge base requirements for passing an elementary science test

By Peter Clark, Philip Harrison, Niranjan Balasubramanian. 2013

48 citations recorded by the source corpus.

Our long-term interest is in machines that contain large amounts of general and scientific knowledge, stored in a "computable" form that supports reasoning and explanation. As a medium-term focus for this, our goal is to have the computer pass a fourth-grade science test, anticipating that much of the required knowledge will need to be acquired semi-automatically. This paper presents the first step towards this goal, namely a blueprint of the knowledge requirements for an early science exam, and a brief description of the resources, methods, and challenges involved in the semi-automatic acquisition of that knowledge. The result of our analysis suggests that as well as fact extraction from text and statistically driven rule extraction, three other styles of automatic knowledge base construction (AKBC) would be useful: acquiring definitional knowledge, direct 'reading' of rules from texts that state them, and, given a particular representational framework (e.g., qualitative reasoning), acquisition of specific instances of those models from text (e..g, specific qualitative models).

Subfields: Artificial Intelligence, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Question Answering & Reading Comprehension

Citations in the corpus

Long Short-Term Memory-Networks for Machine Reading (2016, background): “The term machine reading is related to a wide range of tasks from answering reading comprehension questions (Clark et al., 2013), to fact and relation extraction (Etzioni et al.”
Crowdsourcing Multiple Choice Science Questions (2017, background): “Science exam QA is a high-level NLP task which requires the mastery and integration of information extraction, reading comprehension and common sense reasoning (Clark et al., 2013; Clark, 2015).”
WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference (2018, background): “…proposed as a challenge task for question answering (Clark, 2015), as these questions have very challenging knowledge and inference requirements (Clark et al., 2013; Jansen et al., 2016), but are expressed in simple-enough language that the linguistic challenges are likely surmountable in the…”
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset (2018, background): “3 Multiple surveys of the knowledge base requirements for accomplishing this task (Clark et al., 2013; Jansen et al., 2016) concluded that advanced inference methods were necessary for many of the questions, as they could not be answered by simple fact-based retrieval.”
Elementary School Science and Math Tests as a Driver for AI: Take the Aristo Challenge! (2015, neutral): “For a more detailed analysis, see (Clark et al., 2013).”
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project (2019, background): “In particular: handling natural language became a central focus; Most knowledge was to be acquired automatically (not manually); Machine learning was to play a central role; questions were to be answered exactly as written; and the project restarted at elementary-level science (rather than college-l”
Answering Science Exam Questions Using Query Rewriting with Background Knowledge (2018, background): “A survey of the knowledge base requirements for answering questions from early science questions was performed by Clark, Harrison, and Balasubramanian (2013).”
The Limitations of Standardized Science Tests as Benchmarks for Artificial Intelligence Research: Position Paper (2014, background): “Clark, Harrison, and Balasubramanian (2013) propose a project of passing the New York State Regents Science Test for 4th graders.”
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2019, background): “This choice is motivated by fact that the science domain is considered particularly challenging for QA (Clark et al., 2013; Clark, 2015; Clark et al., 2019), and existing science benchmarks are known to involve widespread use of such knowledge (see Clark et al. (2018); Boratko et al. (2018) for…”
Prerequisite Skills for Reading Comprehension: Multi-Perspective Analysis of MCTest Datasets and Systems (2017, background): “Mathematical operations consist of four arithmetic operations, more advanced mathematics, and geometric comprehension as in Clark, Harrison, and Balasubramanian (2013) and Clark (2015)3.”
Alignment over Heterogeneous Embeddings for Question Answering (2019, methodology): “To select the correct answer in such complex questions, especially for short questions, a successful method would have to incorporate inference, e.g., recognizing process questions such as the one in the ﬁgure and coupling with it with a dedicated problem solving method (Clark et al., 2013).”
What’s in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams (2016, methodology): “Clark et al. (2013) compiled an initial analysis of the questions in these datasets, and identiﬁed 7 broad categories of knowledge and inference requirements.”
Combining Natural Logic and Shallow Reasoning for Question Answering (2016, methodology): “We evaluate our entailment system on the Regents Science Exam portion of the Aristo dataset (Clark et al., 2013; Clark, 2015).”
How to Write Science Questions that Are Easy for People and Hard for Computers (2016, background): “Clark, Harrison, and Balasubramanian (2013) propose a project of passing the New York State Regents Science Test for 4th graders.”
Fact Checking: Task definition and dataset construction (2014, background): “This desideratum is present in other tasks such as the recently proposed science test question answering (Clark et al., 2013).”
VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases (2015, methodology): “To demonstrate this, we ran a preliminary experiment using our approach on a small subset of the NewYork Regents’ 4th grade science exam [7] that were visually relevant.”