Research priorities for robust and beneﬁcial artiﬁcial intelligence

Subfields: Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

Citations in the corpus

Artificial Intelligence, Values, and Alignment (2020, background): “For example, if powerful AI systems function at super-human speed, which seems likely, then it may not be possible to provide them with immediate and continuous direction in this way (Russell et al., 2015; Soares, 2015).”
Scalable agent alignment via reward modeling: a research direction (2018, methodology): “We build on taxonomies and problem deﬁnitions from many authors before us, highlighting tractable and neglected problems in the ﬁeld of AI safety (Russell et al., 2015; Soares, 2015; Amodei et al., 2016; Taylor et al., 2016; Soares & Fallenstein, 2017; Christiano, 2017; Leike et al., 2017; Ortega…”
AI safety via debate (2018, background): “Learning to align an agent’s actions with the values and preferences of humans is a key challenge in ensuring that advanced AI systems remain safe [Russell et al., 2016].”
Concrete Problems in AI Safety (2016, background): “[128] propose research priorities for robust and beneﬁcial artiﬁcial intelligence, and includes several other topics in addition to AI-related accidents, though it also discusses accidents .”
The Social Impact of Natural Language Processing (2016, background): “Steps towards this have been taken by Russell et al. (2015).”
Unsolved Problems in ML Safety (2021, background): “An early work that helps identify safety problems is Russell et al. , 2015 [154], who identify many potential”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “…and inclusion in AI systems [13], ﬁndings that some automated systems may reinforce inequality and bias [14], and requirements for safe and secure AI in safety-critical tasks [15], there has been a recent explosion of interest in interpreting the representations and decisions of black-box models.”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, background): “Ensuring that the agents behave in alignment with human values is known, appropriately, as the value alignment problem [Amodei et al., 2016, Hadﬁeld-Menell et al., 2016, Russell et al., 2015, Bostrom, 2014a, Leike et al., 2017].”
AI Safety Gridworlds (2017, background): “…field of AI safety still lacks a general consensus on its research problems, and there have been several recent efforts to turn these concerns into technical problems on which we can make direct progress (Soares and Fallenstein, 2014; Russell et al., 2015; Taylor et al., 2016; Amodei et al., 2016).”
An approach to reachability analysis for feed-forward ReLU neural networks (2017, background): “This has been compounded by recent public calls for the development of “responsi-ble” and “veriﬁable AI” [27].”
AGI Safety Literature Review (2018, background): “Russell et al. (2016) and the Future of Life Institute (FLI) take the broadest view, covering societal and technical challenges in both the near and the long term future.”
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning (2018, background): “…and inclusion in AI systems [13], ﬁndings that some automated systems may reinforce inequality and bias [14], and requirements for safe and secure AI in safety-critical tasks [15], there has been a recent explosion of interest in interpreting the representations and decisions of black-box models.”
Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text (2020, background): “This is often referred to as the value alignment problem [Soares and Fallenstein, 2014; Russell et al. , 2015; Arnold et al. , 2017; Abel et al. , 2016].”
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure (2018, methodology): “Applications include uses of automated perception and classiﬁcation in high-stakes decisions with signiﬁcant consequences for people and broader society (Dietterich and Horvitz 2015; Russell, Dewey, and Tegmark 2015; Salay, Queiroz, and Czarnecki 2017; Amodei et al. 2016).”
Should Robots be Obedient? (2017, background): “Highly capable autonomous system that don’t obey human commands run substantially higher risks, ranging from property damage to loss of life (Asaro, 2006; Lewis, 2014) to potentially catastrophic threats to humanity (Bostrom, 2014; Russell et al. , 2015).”
Alignment for Advanced Machine Learning Systems (2020, background): “For a more general overview of research questions in AI safety, including both strictly near-term and strictly long-term issues in computer science and other disciplines, see Russell, Dewey, and Tegmark (2015).”
Interpretable Multi-Objective Reinforcement Learning through Policy Orchestration (2018, background): “These topics have gained popularity among a broad community including philosophers (Wallach and Allen 2008) and non-proﬁts (Russell, Dewey, and Tegmark 2015).”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, background): “The question of whether and why we should create general AI is a complicated one and is the focus of many articles [143, 41, 4, 18, 15].”
Reducing Non-Normative Text Generation from Language Models (2020, background): “This is referred to as the value alignment problem 1 https://www.newsweek.com/amazon-echo% 2Dtells-uk-woman-stab-herself-1479074/ (Soares and Fallenstein, 2014; Russell et al., 2015; Arnold et al., 2017; Abel et al., 2016).”