Low Impact Artificial Intelligences

By Stuart Armstrong, Benjamin Levinstein, Benjamin Levinstein. 2017

37 citations recorded by the source corpus.

There are many goals for an AI that could become dangerous if the AI becomes superintelligent or otherwise powerful. Much work on the AI control problem has been focused on constructing AI goals that are safe even for such AIs. This paper looks at an alternative approach: defining a general concept of `low impact'. The aim is to ensure that a powerful AI which implements low impact will not modify the world extensively, even if it is given a simple or dangerous goal. The paper proposes various ways of defining and grounding low impact, and discusses methods for ensuring that the AI can still be allowed to have a (desired) impact despite the restriction. The end of the paper addresses known issues with this approach and avenues for future research.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability, Philosophy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tom Everitt
2. Shane Legg
3. Jan Leike
4. Richard Ngo
5. Miljan Martic
6. David Krueger
7. Victoria Krakovna
8. Laurent Orseau

In Safety Research

1. Richard Ngo
2. Shane Legg
3. Pedro A. Ortega
4. Laurent Orseau
5. Tom Everitt
6. Victoria Krakovna
7. Miljan Martic

In Safety, Robustness & Interpretability

1. Tom Everitt
2. Shane Legg
3. Victoria Krakovna
4. Miljan Martic
5. Pedro A. Ortega
6. Jan Leike
7. Richard Ngo
8. David Krueger

In Philosophy

1. David Krueger
2. Pedro A. Ortega
3. Laurent Orseau
4. Tom Everitt
5. Richard Ngo
6. Victoria Krakovna
7. Miljan Martic
8. Jan Leike

Citations in the corpus

• Scalable agent alignment via reward modeling: a research direction (2018, background): “• Side-effects (Armstrong & Levinstein, 2017; Zhang et al., 2018b; Krakovna et al., 2018): the agent is not incentivized to reduce effects unrelated to its main objectives, even if those are irreversible or difficult to reverse.”
• AI Safety Gridworlds (2017, background): “Armstrong and Levinstein (2017) define a distance metric between states by measuring differences in a large number of variables, and penalize the distance from the inaction baseline.”
• Avoiding Side Effects By Considering Future Tasks (2020, background): “For example, Armstrong and Levinstein [3] define doing nothing as equivalent to switching off the agent, which is not straightforward to represent as a policy in environments without a given noop action.”
• Measuring and avoiding side effects using relative reachability (2018, methodology): “The two main types of approaches in the literature, ergodicitypreserving safe exploration (Moldovan and Abbeel, 2012; Eysenbach et al., 2017) and low impact (Armstrong and Levinstein, 2017; Taylor et al., 2016), use criteria of this form.”