Teachable robots: Understanding human teaching behavior to build more effective robot learners

By Andrea Lockerd Thomaz, Cynthia Lynn Breazeal. 2008

450 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Social Psychology, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Social Psychology

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Scalable agent alignment via reward modeling: a research direction (2018, background): “Humans generally do poorly at training RL agents by providing scalar rewards directly; often they teach a shaped reward function and provide rewards that depend on the agent’s policy (Thomaz & Breazeal, 2008; MacGlashan et al., 2017).”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, background): “Thomaz and Breazeal pioneered the effort in understanding human behavior in teaching learning agents [ 2008 ] .”
Learning from Physical Human Corrections, One Feature at a Time (2018, background): “Indeed, sequential teaching may come more naturally to people collaborating with robots [21, 22], and reduces the burden on users to coordinate all aspects of the task simultaneously during each individual correction.”
Interactive Learning from Policy-Dependent Human Feedback (2017, background): “This failure has led to the development of new reinforcement-learning algorithms that are designed to learn from human-generated rewards and investigations into how people give interactive feedback [1, 2, 3, 4, 5, 6].”
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback (2017, background): “Non-expert raters are skewed both for reinforcement learning (Thomaz et al., 2006; Thomaz and Breazeal, 2008; Loftin et al., 2014) and recommender systems (Herlocker et al.”
Active Reward Learning (2014, neutral): “Unfortunately, as already shown by Thomaz and Breazeal [35] and by Cakmak and Thomaz [5], humans have considerable”
Power to the People: The Role of Humans in Interactive Machine Learning (2014, methodology): “In the experiment by Thomaz and Breazeal (2008) users trained a simulated agent to bake a cake through a series of object manipulations.”
Society-in-the-loop: programming the algorithmic social contract (2017, background): “This includes dynamically adapting the degree of autonomy given to robots [19, 73], interactively teaching reinforcement learning robots to adopt particular behaviors [74], and designing ﬂexible human-robot teams [38].”
Robots That Use Language (2020, background): “A bottle consists of a bottle and a cap, and a person referencing “the bottle” may mean both, or they may say, “Grab the bottle and then turn the cap,”referring”
Safe Reinforcement Learning via Shielding (2017, background): “In each time step, the human teacher tunes the reward signal before sending it to the agent [19,20].”
Learning Visual Object Categories for Robot Affordance Prediction (2010, unknown): “(2008), Thomaz and Breazeal (2008), Rybski et al. (2007), Lopes et al. (2007), Bentivegna et al. (2006), Bernardin et al. (2005), and Jenkins and Matarić (2004).”
Learning non-myopically from human-generated reward (2013, background): “This mapping to 1 and -1, though not infallible, is an intuitive choice that is similar to that of related works that explain their exact mappings [14, 13, 10, 11].”
PERCEPTUAL GOAL SPECIFICATIONS FOR REINFORCEMENT LEARNING (2017, background): “For example, some approaches allow humans to provide rewards to agents in real-time [27, 35, 60].”
Learning about objects with human teachers (2009, result): “We have shown examples of this in prior work [18].”
A comprehensive survey on safe reinforcement learning (2015, methodology): “Thomaz and Breazeal (2006, 2008) also introduce an interface between the human teacher and the learner agent.”