Modeling AGI Safety Frameworks with Causal Influence Diagrams
By Tom Everitt, Ramana Kumar, Victoria Krakovna, Shane Legg. 2019
24 citations recorded by the source corpus.
Proposals for safe AGI systems are typically made at the level of frameworks, specifying how the components of the proposed system should be trained and interact with each other. In this paper, we model and compare the most promising AGI safety frameworks using causal influence diagrams. The diagrams show the optimization objective and causal assumptions of the framework. The unified representation permits easy comparison of frameworks and their assumptions. We hope that the diagrams will serve as an accessible and visual introduction to the main AGI safety frameworks.
Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Tom Everitt
- Iason Gabriel
- Shane Legg
- Geoffrey Irving
- Pedro A. Ortega
- Vladimir Mikulik
- Eric Langlois
- Zachary Kenton
In Safety Research
- Iason Gabriel
- Shane Legg
- Pedro A. Ortega
- Ryan Carey
- Eric Langlois
- Vladimir Mikulik
- Geoffrey Irving
- Laura Weidinger
In Safety, Robustness & Interpretability
- Tom Everitt
- Shane Legg
- Eric Langlois
- Iason Gabriel
- Pedro A. Ortega
- Vladimir Mikulik
- Ryan Carey
- Geoffrey Irving
Citations in the corpus
- Alignment of Language Agents (2021, background): “…stance towards the agent (Dennett, 1989), which means it’s not important whether the agent’s primary objective has an obvious physical correlate, as long as treating the system as an agent optimizing for that primary objective is a good model for predicting its behaviour (Everitt et al., 2019a).”
- Agent Incentives: A Causal Perspective (2021, methodology): “8Since a preprint of this paper was placed online (Everitt et al. 2019c), this completeness result was independently discovered by Zhang, Kumor, and Bareinboim (2020, Thm. 2) and Lee and Bareinboim (2020, Thm. 1).”