Multi-Armed Bandits for Intelligent Tutoring Systems

By Benjamin Clement, Didier Roy, Pierre-Yves Oudeyer, Manuel Lopes. 2013

154 citations recorded by the source corpus.

We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity which makes them progress faster. We introduce two algorithms that rely on the empirical estimation of the learning progress, RiARiT that uses information about the difficulty of each exercise and ZPDES that uses much less knowledge about the problem. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by specific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this optimization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the expert and allowing the system to deal with didactic gaps in its knowledge. The system is evaluated in a scenario where 7-8 year old schoolchildren learn how to decompose numbers while manipulating money. Systematic experiments are presented with simulated students, followed by results of a user study across a population of 400 school children.

Subfields: Artificial Intelligence, Computer Science Applications, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Management Science and Operations Research

Citations in the corpus

Automated Curriculum Learning for Neural Networks (2017, methodology): “We take inspiration from a previous work that modelled an adaptive student with a multiarmed bandit in the context of developmental learning (Lopes and Oudeyer, 2012; Clement et al., 2015).”
Bootstrapped Policy Gradient for Difficulty Adaptation in Intelligent Tutoring Systems (2019, methodology): “In these large action space cases, Multi-armed Bandit (MAB) or contextual MAB frameworks are usually employed as more scalable approaches [5, 10, 12].”
Tackling Climate Change with Machine Learning (2019, methodology): “s have traditionally been deﬁned and constructed by hand, recent approaches have applied ML techniques such as multi-armed bandit techniques to adaptively personalize sequences of learning activities [796], LSTMs to generate questions to evaluate language comprehension [797], and reinforcement learn”
Mining Big Data in Education: Affordances and Challenges (2020, methodology): “These methods have been used to discover how best to sequence practice problems by testing out many different sequences with large numbers of observations from each student (Clement et al., 2015).”
Teacher–Student Curriculum Learning (2017, unknown): “It has been successfully applied in the context of developmental robotics to learn object manipulation [18], [19] and also in actual classroom settings to teach primary school students [20].”