AI Safety Needs Social Scientists

By G. Irving, Amanda Askell. 2019

59 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Finding Generalizable Evidence by Learning to Convince Q&A Models (2019, background): “Debate Evidence extraction can be viewed as a form of debate, in which multiple agents support different stances (Irving et al., 2018; Irving and Askell, 2019).”
Alignment of Language Agents (2021, background): “Whether debate works in practice with real humans is an open question (Irving and Askell, 2019).”
Measuring Progress on Scalable Oversight for Large Language Models (2022, background): “• Debate: Techniques in this family (Irving et al., 2018; Irving and Askell, 2019) adapt a model (potentially but not necessarily a natural-language assistant) to propose an answer to a question and then alternately play the role of two participants in a debate, surfacing and critiquing arguments…”
GPQA: A Graduate-Level Google-Proof Q&A Benchmark (2023, neutral): “Irving & Askell (2019) give nine desiderata for datasets suitable for scalable oversight experiments.”
Large Language Model Alignment: A Survey (2023, background): “Debate (Irving et al., 2018; Irving and Askell, 2019; Du et al., 2023) is another promising scalable oversight paradigm that can not only achieve single-agent alignment but also enable multi-agent alignment.”