Managing extreme AI risks amid rapid progress

By Y. Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Trevor Darrell, Y. Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz and 15 more. Science, 2023

431 citations recorded by the source corpus.

Preparation requires technical research and development, as well as adaptive, proactive governance Artificial intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI’s impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI (1), there is a lack of consensus about how to manage them. Society’s response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness and barely address autonomous systems. Drawing on lessons learned from other safety-critical technologies, we outline a comprehensive plan that combines technical research and development (R&D) with proactive, adaptive governance mechanisms for a more commensurate preparation.

Subfields: Artificial Intelligence, Health Informatics, Safety Research

View Paper: https://www.semanticscholar.org/paper/2996da066a48e4674454f5f75ad5cc6f85f23ff6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Quoc V. Le
  4. Percy Liang
  5. Mohit Bansal
  6. Ruslan Salakhutdinov
  7. Yoshua Bengio
  8. Jianfeng Gao

In Safety Research

  1. Adam Lerer
  2. Ofir Nachum

Citations in the corpus