Addressing the assessment challenge with an online system that tutors as it assesses

By Mingyu Feng, Neil Heffernan III, K. Koedinger. User modeling and user-adapted interaction, 2009

560 citations recorded by the source corpus.

Secondary teachers across the United States are being asked to use formative assessment data (Black and Wiliam 1998a,b; Roediger and Karpicke 2006) to inform their classroom instruction. At the same time, critics of US government’s No Child Left Behind legislation are calling the bill “No Child Left Untested”. Among other things, critics point out that every hour spent assessing students is an hour lost from instruction. But, does it have to be? What if we better integrated assessment into classroom instruction and allowed students to learn during the test? We developed an approach that provides immediate tutoring on practice assessment items that students cannot solve on their own. Our hypothesis is that we can achieve more accurate assessment by not only using data on whether students get test items right or wrong, but by also using data on the effort required for students to solve a test item with instructional assistance. We have integrated assistance and assessment in the ASSISTment system. The system helps teachers make better use of their time by offering instruction to students while providing a more detailed evaluation of student abilities to the teachers, which is impossible under current approaches. Our approach for assessing student math proficiency is to use data that our system collects through its interactions with students to estimate their performance on an end-of-year high stakes state test. Our results show that we can do a reliably better job predicting student end-of-year exam scores by leveraging the interaction data, and the model based on only the interaction information makes better predictions than the traditional assessment model that uses only information about correctness on the test items.

Subfields: Artificial Intelligence, Computer Science Applications, Education

View Paper: https://www.semanticscholar.org/paper/c47a3ffd82627bf715333c25ddaa51c1956fa204

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Education

Citations in the corpus

Deep Knowledge Tracing (2015, methodology): “Benchmark Dataset : In order to understand how our model compared to other models we evaluated models on the Assistments 2009-2010 public benchmark dataset [11].”
Knowledge Tracing: A Survey (2022, methodology): “The ASSISTments datasets [32, 84] contain longitudinal data collected from the free online tutoring ASSISTment platform.”
The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning (2012, background): “For example, student performance in the ASSISTment computer tutoring system (Feng et al., 2009) reveals that errors in simple fraction multiplication word problems (e.g., ''What is 3 ⁄ 4 of 1 ⁄ 2?'') are sometimes not about the math, but about vocabulary––follow-up questions indicate that students…”
Neural Cognitive Diagnosis for Intelligent Education Systems (2019, methodology): “ASSIST (ASSISTments 2009-2010 ”skill builder” ) is an open dataset collected by the AS-SISTments online tutoring systems (Feng, Heffernan, and Koedinger 2009), which only provides student response logs and knowledge concepts 2 .”
Graph-based knowledge tracing: Modeling student proficiency using graph neural networks (2019, unknown): “We used the following four datasets [9] for our experiments.”
Dynamic Key-Value Memory Networks for Knowledge Tracing (2016, methodology): “ASSISTments2009: This dataset [5] is gathered from the ASSISTments online tutoring platform.”
Enhancing the prediction of student performance based on the machine learning XGBoost algorithm (2021, methodology): “ASSISTments 2009 − 2010 : is a real dataset extracted from the ASSISTments platform (Feng et al., 2009).”