The alignment problem from a deep learning perspective

By Richard Ngo. 2022

333 citations recorded by the source corpus.

In coming years or decades, artificial general intelligence (AGI) may surpass human capabilities across many critical domains. We argue that, without substantial effort to prevent it, AGIs could learn to pursue goals that are in conflict (i.e. misaligned) with human interests. If trained like today's most capable models, AGIs could learn to act deceptively to receive higher reward, learn misaligned internally-represented goals which generalize beyond their fine-tuning distributions, and pursue those goals using power-seeking strategies. We review emerging evidence for these properties. In this revised paper, we include more direct empirical evidence published as of early 2025. AGIs with these properties would be difficult to align and may appear aligned even when they are not. Finally, we briefly outline how the deployment of misaligned AGIs might irreversibly undermine human control over the world, and we review research directions aimed at preventing this outcome.

Subfields: Computer Vision and Pattern Recognition, Alignment & RLHF

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Bernt Schiele
  3. Dacheng Tao
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Xiaogang Wang
  8. Andrew Zisserman

In Alignment & RLHF

  1. Jacob Hilton
  2. Ethan Perez
  3. Anca Dragan
  4. John Schulman
  5. Tomasz Korbak
  6. Leo Gao
  7. J'er'emy Scheurer
  8. Jeff Wu

Citations in the corpus