Learning What to Value

By Dan Dewey. 2011

115 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Scalable agent alignment via reward modeling: a research direction (2018, methodology): “The ﬁrst formal treatment of the agent alignment problem is due to Dewey (2011) and has since been reﬁned (Hadﬁeld-Menell et al., 2016; Everitt & Hutter, 2018).”
Concrete Problems in AI Safety (2016, background): “n of an abstract objective function, because there are certain sequences of actions for which the objective function is physically replaced. This particular failure mode is often called “wireheading” [48, 125, 41, 65]. It is particularly concerning in cases where a human may be in the reward loop, g”
Model-based Utility Functions (2011, background): “Dewey (2011) argues that human-designed reinforcement-learning agents will alter their environments to get rewards regardless of whether they achieve the goals of their human designers.”
Avoiding Unintended AI Behaviors (2012, background): “The AGI-11 conference included three papers (Orseau and Ring 2011; Ring and Orseau 2011; Dewey 2011) that employed the mathematics of rational agents to analyze ways that AI agents may fail to satisfy the intentions of their designers.”
Alignment for Advanced Machine Learning Systems (2020, background): “The problem of generalizable environmental goals is related to the problem of reward hacking, which has been discussed by Dewey (2011) and Amodei et al. (2016), wherein an AI system takes control of the physical mechanism that dispenses reward and alters it.”
Informatic parcellation of the network involved in the computation of subjective value. (2014, background): “This area has had several wide classes of computations attributed to it, including decision and strategic control (Venkatraman et al., 2009), environment volatility (Behrens et al., 2007) and multiplexing of decision parameters (Kennerley et al., 2009; Hayden and Platt, 2010; Hare et al., 2011b).”
Utility function security in artificially intelligent agents (2014, background): “(Dewey, 2011) Hibbard demonstrates a mathematical justiﬁcation of why the agents will not choose to self-modify and contends, Our belief in external reality is so strong that when it conﬂicts with our perceptions, we often seek to explain the conﬂict by some error in our perceptions.”
Motivated Value Selection for Artificial Agents (2015, methodology): “It has thus been suggested that agents not be fully programmed from the very beginning, but instead use techniques, analogous to machine learning, to learn values during development (Dewey 2011; Goertzel and Pitt 2012).”
Defining and Characterizing Reward Hacking (2022, unknown): “One source of confusion and disagreement is the role of the reward function; it is variously considered as a means of specifying a task (Leike et al., 2018) or encoding broad human values (Dewey, 2011); such distinctions are discussed by Christiano (2019) and Gabriel (2020).”
Embedded Agency (2019, background): “Daniel Dewey (2011) calls the second sort of agent an observation-utility max-imizer .”
Self-Modification of Policy and Utility Function in Rational Agents (2016, background): “However, it is nonetheless important to develop a theory for their control (Bostrom, 2014).”
Avoiding Wireheading with Value Reinforcement Learning (2016, background): “Other value learning suggestions have been speculative or vague (Bostrom, 2014a,b; Dewey, 2011).”
The Off-Switch Game (2016, background): “Dewey (2011) proposed the idea of an agent as learning an uncertain utility function and argues that it avoid some of the pitfalls (e.g., wireheading) that reinforcement learning agents are susceptible to.”