Human Compatible: Artificial Intelligence and the Problem of Control

By Stuart Russell. 2019

918 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Alignment of Language Agents (2021, background): “In this paper we focus on behavioural issues that arise due to a lack of alignment, where the system does not do what we intended it to do (Bostrom, 2014; Christiano, 2018; Leike et al., 2018; Russell, 2019).”
The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, background): “As reinforcement learning agents are trained with better algorithms, more data, and larger policy models, they are at increased risk of overﬁtting their objectives (Russell, 2019).”
A Generalist Agent (2022, background): “For this reason, preference learning, uncertainty modeling and value alignment (Russell, 2019) are especially important for the design of human-compatible generalist agents.”
RL with KL penalties is better viewed as Bayesian inference (2022, background): “The second line of work is concerned more broadly with ensuring that objectives that machine learning systems pursue are aligned with human values (Amodei et al., 2016; Russell, 2019).”
Aligning AI Optimization to Community Well-Being (2020, background): “Dynamism also avoids the serious problems that can arise from over-optimization for a single objective, such as a robot which injures humans in an attempt to fetch a coffee more quickly (Russell, 2019).”
Can AI language models replace human participants? (2023, neutral): “Language models are trained on many people, but then tend to collapse the diversity of judgments into a single modal opinion [15].”
Artificial Intelligence and Marketing: Pitfalls and Opportunities (2020, background): “At the very early design stages, marketing organizations need to put in place mechanisms to control, stop, and override AI algorithms in real-time (Armstrong, Sandberg, & Borstrom, 2012; Russell 2019).”
Consequences of Misaligned AI (2021, background): “Value Alignment The importance of having AI objectives line up with human objectives has been well-documented, with numerous experts postulating that misspecified goals can lead to undesirable results (29; 27; 8; 28).”
Agent Incentives: A Causal Perspective (2021, background): “For example, Russell (2019) has hypothesised that an advanced AI system would seek to manipulate its objective function (or human overseer) to obtain reward.”
The Next Generation of Research on IS Use: A Theoretical Framework of Delegation to and from Agentic IS Artifacts (2021, methodology): “learn, adapt, act autonomously, and be aware of the need to act without being prompted by users (e.g., Floreano and Wood 2015; Russell 2019).”
The Ghost in the Machine has an American accent: value conflict in GPT-3 (2022, background): “There is room for increased methodological diversity into the human alignment in AI problem using diverse sociocultural, philosophical, and linguistic perspectives[44, 45], notably in a global pluralist setting[11].”
Why AI is harder than we think (2021, background): “Stuart Russell, co-author of a widely used textbook on AI, predicts that “superintelligent AI” will “probably happen in the lifetime of my children” [23] and Sam Altman, CEO of the AI company OpenAI, predicts that within decades, computer programs “will do almost everything, including making new…”
Theory of Mind and Preference Learning at the Interface of Cognitive Science, Neuroscience, and AI: A Review (2022, background): “Rather than coding machines in order for them to optimize a certain objective (which, if misspeciﬁed, can lead to failure), Russell (2019) proposes that we aim for machines to be beneﬁcial to humans: “Machines are beneﬁcial to the extent that their actions can be expected to achieve our…”
Rethinking Formal Models of Partially Observable Multiagent Decision Making (2019, background): “8For example, a thief might steal a wallet without immediately inspecting its contents, or a robot might be optimizing for my preferences while having uncertainty over them [35].”
Are aligned neural networks adversarially aligned? (2023, background): “Some alignment researchers [Russell, 2019, Bucknall and Dori-Hacohen, 2022, Ngo, 2022, Carl-smith, 2022] believe that sufficiently advanced language models should be aligned to prevent an existential risk [Bostrom, 2013] to humanity: if this were true, an attack that causes such a model to become…”