Research Priorities for Robust and Beneficial Artificial Intelligence

By Stuart J. Russell, Dan Dewey, Max Tegmark. 2015

735 citations recorded by the source corpus.

Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls. This article gives numerous examples (which should by no means be construed as an exhaustive list) of such worthwhile research aimed at ensuring that AI remains robust and beneficial.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Scalable agent alignment via reward modeling: a research direction (2018, methodology): “We build on taxonomies and problem deﬁnitions from many authors before us, highlighting tractable and neglected problems in the ﬁeld of AI safety (Russell et al., 2015; Soares, 2015; Amodei et al., 2016; Taylor et al., 2016; Soares & Fallenstein, 2017; Christiano, 2017; Leike et al., 2017; Ortega…”
Unsolved Problems in ML Safety (2021, background): “, 2015 [152], who identify many potential avenues for safety, spanning robustness, machine ethics, research on AI’s economic impact, and more.”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “reinforce inequality and bias [14], and requirements for safe and secure AI in safety-critical tasks [15], there has been a recent explosion of interest in interpreting the representations”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, background): “Ensuring that the agents behave in alignment with human values is known, appropriately, as the value alignment problem [Amodei et al., 2016, Hadﬁeld-Menell et al., 2016, Russell et al., 2015, Bostrom, 2014a, Leike et al., 2017].”
AI Safety Gridworlds (2017, background): “This nascent field of AI safety still lacks a general consensus on its research problems, and there have been several recent efforts to turn these concerns into technical problems on which we can make direct progress (Soares and Fallenstein, 2014; Russell et al., 2015; Taylor et al., 2016; Amodei et”
An approach to reachability analysis for feed-forward ReLU neural networks (2017, background): “This has been compounded by recent public calls for the development of “responsible” and “verifiable AI” [27].”
Ethics of artificial intelligence and robotics (2020, background): “…discussion: (Häggström 2016) provides an overview of high-impact technological risk for humanity; (Tegmark 2017) focuses on AI and human life ‘3.0’ after singularity while (Russell, Dewey, and Tegmark 2015) and (Bostrom, Dafoe, and Flynn forthcoming) survey longer-term policy issues in ethical AI.”
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure (2018, methodology): “Applications include uses of automated perception and classiﬁcation in high-stakes decisions with signiﬁcant consequences for people and broader society (Dietterich and Horvitz 2015; Russell, Dewey, and Tegmark 2015; Salay, Queiroz, and Czarnecki 2017; Amodei et al. 2016).”
Meaningful Human Control over Autonomous Systems: A Philosophical Account (2018, background): “Scientists, entrepreneurs, policy-makers, and NGO workers involved in these initiatives agree that in order to prevent future robots from negatively impacting human society, we need to immediately start a systematic reflection on the ethical principles for the regulation and design of autonomous wea”
Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text (2020, background): “This is often referred to as the value alignment problem [Soares and Fallenstein, 2014; Russell et al. , 2015; Arnold et al. , 2017; Abel et al. , 2016].”
Semantics Derived Automatically from Language Corpora Contain Human-like Moral Choices (2019, neutral): “And machine ethics [2, 11, 19] is no exception.”
Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes (2020, background): “Since then, a number of groups have suggested research priorities for building safe and friendly intelligent systems (Russell, Dewey, and Tegmark 2015; Amodei et al. 2016).”
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning (2018, background): “With impending regulations like the European Union’s “Right to Explanation” [12], calls for diversity and inclusion in AI systems [13], findings that some automated systems may reinforce inequality and bias [14], and requirements for safe and secure AI in safety-critical tasks [15], there has been a”