Pragmatic-Pedagogic Value Alignment

By Jaime Fernández Fisac, Monica A. Gates, Jessica B. Hamrick, Chang Liu, Dylan Hadfield-Menell, Malayandi Palaniappan, Dhruv Malik, S. Sastry, T. Griffiths, Anca Dragan. 2017

92 citations recorded by the source corpus.

As intelligent systems gain autonomy and capability, it becomes vital to ensure that their objectives match those of their human users; this is known as the value-alignment problem. In robotics, value alignment is key to the design of collaborative robots that can integrate into human workflows, successfully inferring and adapting to their users' objectives as they go. We argue that a meaningful solution to value alignment must combine multi-agent decision theory with rich mathematical models of human cognition, enabling robots to tap into people's natural collaborative capabilities. We present a solution to the cooperative inverse reinforcement learning (CIRL) dynamic game based on well-established cognitive models of decision making and theory of mind. The solution captures a key reciprocity relation: the human will not plan her actions in isolation, but rather reason pedagogically about how the robot might learn from them; the robot, in turn, can anticipate this and interpret the human's actions pragmatically. To our knowledge, this work constitutes the first formal analysis of value alignment grounded in empirically validated cognitive models.

Subfields: Artificial Intelligence, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Human-in-the-Loop Imitation Learning using Remote Teleoperation (2020, background): “Other paradigms for human-in-the-loop policy learning include collaboration [14, 15], teaching the robot through informative sample selection [7, 9, 17], and leveraging physical kinesthetic corrections [5, 6].”
Machine Theory of Mind (2018, background): “They will also be highly useful for communication and pedagogy (Dragan et al., 2013; Fisac et al., 2017; Milli et al., 2017), and will thus likely play a key role in human-machine interaction.”
Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences (2020, background): “Safe imitation learning is closely related to the problem of value alignment, which seeks to design methods that prevent AI systems from acting in ways that violate human values (Hadﬁeld-Menell et al., 2016; Fisac et al., 2020).”