Aligning AI With Shared Human Values

By Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Zheng Li, Dawn Song, Jacob Steinhardt. 2020

970 citations recorded by the source corpus.

We show how to assess a language model's knowledge of basic concepts of morality. We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality. Models predict widespread moral judgments about diverse text scenarios. This requires connecting physical and social world knowledge to value judgements, a capability that may enable us to steer chatbot outputs or eventually regularize open-ended reinforcement learning agents. With the ETHICS dataset, we find that current language models have a promising but incomplete understanding of basic ethical knowledge. Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.

Subfields: Artificial Intelligence, Safety Research, Alignment & RLHF, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Luke S. Zettlemoyer
  2. Mohit Bansal
  3. Percy Liang
  4. Yejin Choi
  5. Jianfeng Gao
  6. Noah A. Smith
  7. Graham Neubig
  8. Hannaneh Hajishirzi

In Safety Research

  1. Richard Ngo
  2. Yejin Choi
  3. William S. Isaac
  4. Iason Gabriel
  5. Amanda Askell
  6. Dawn Song
  7. Bo Li
  8. Ruslan Salakhutdinov

In Alignment & RLHF

  1. Geoffrey Irving
  2. Richard Ngo
  3. Jacob Steinhardt
  4. John Schulman
  5. Yejin Choi
  6. Iason Gabriel
  7. Amanda Askell
  8. Daniel Klein

In Safety, Robustness & Interpretability

  1. Jacob Steinhardt
  2. Percy Liang
  3. Dawn Song
  4. Dan Hendrycks
  5. Nicholas Carlini
  6. Bo Li
  7. Adina Williams
  8. Yulia Tsvetkov

Citations in the corpus