Moving beyond the Turing Test with the Allen AI Science Challenge

By Carissa Schoenick, Peter Clark, Oyvind Tafjord, Peter D. Turney, Oren Etzioni. 2016

57 citations recorded by the source corpus.

Answering questions correctly from standardized eighth-grade science tests is itself a test of machine intelligence.

Subfields: Computer Science Applications, Management Information Systems, Molecular Biology, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Management Information Systems

In Molecular Biology

In Evaluation & Benchmarks

Citations in the corpus

PIQA: Reasoning about Physical Commonsense in Natural Language (2019, background): “Physical understanding is broad domain that touches on everything from scientiﬁc knowledge (Schoenick et al. 2016) to the interactive acquisition of knowledge by embodied agents (Thomason et al. 2016).”
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge (2018, methodology): “Datasets based on human standardized tests have also been used in AI for several years (e.g. Clark et al., 2016; Seo et al., 2014; Fujita et al., 2014; Strickland, 2013), and as part of the NTCIR QALab (NII, 2017), and for the 2016 Allen AI Science Challenge described in the Introduction (Schoenick ”
Crowdsourcing Multiple Choice Science Questions (2017, background): “In a recent Kaggle competition addressing 8 th grade science questions (Schoenick et al., 2016), the highest scoring systems achieved only 60% on a multiple choice test, with retrieval-based systems far outperforming neural systems.”
Deep Learning: A Critical Appraisal (2018, background): “• Scientific reasoning and understanding, as in the Allen AI institute’s 8th grade science challenge (Schoenick, Clark, Tafjord, P, & Etzioni, 2017; Davis, 2016).”
WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference (2018, background): “…al., 2016; Khot et al., 2017; Jansen et al., 2017, inter alia), and top-performing ensemble models nearly reaching a passing grade of 60% on middle school ( 8 th grade) science exams during a recent worldwide competition of 780 teams sponsored by the Allen Institute for AI (Schoenick et al., 2017).”
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project (2019, background): “Most notably, Aristo’s scores on the Regents Exams far exceed earlier performances (e.g., Schoenick et al., 2016; Clark et al., 2016), and represents a new high-point on science questions.”
A strategic framework for artificial intelligence in marketing (2020, background): “Positioning is currently a human task, for it involves judgment, intuition, and creativity that machines are not particularly good at it yet (Davis and Marcus 2015; Schoenick et al. 2017).”
Improving Retrieval-Based Question Answering with Deep Inference Models (2018, background): “[2] have argued that answering science questions with high accuracy can be a good way to assess machine intelligence.”
From Textbooks to Knowledge: A Case Study in Harvesting Axiomatic Knowledge from Textbooks to Solve Geometry Problems (2017, background): “…in solving standardized tests such as reading comprehensions (Richardson et al., 2013; Sachan et al., 2015, inter alia), science question answering (Schoenick et al., 2016; Sachan et al., 2016, inter alia), algebra word problems (Kushman et al., 2014, inter alia), geometry problems (Seo et al.,…”