The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

By Miles Brundage, S. Avin, Jack Clark, H. Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, P. Scharre, Thomas Zeitzoff, Bobby Filar and 16 more. 2018

923 citations recorded by the source corpus.

The following organisations are named on the report: Future of Humanity Institute, University of Oxford, Centre for the Study of Existential Risk, University of Cambridge, Center for a New American Security, Electronic Frontier Foundation, OpenAI. The Future of Life Institute is acknowledged as a funder.

Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety, Robustness & Interpretability

Citations in the corpus

Alignment of Language Agents (2021, background): “• Malicious use (Brundage et al., 2018) of language agents by humans, which can produce disinformation, the spreading of dangerous and/or private information, and discriminatory and harmful content.”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Furthermore, malicious actors [17] have historically worked to circumvent safety controls.”
Unsolved Problems in ML Safety (2021, background): “[23] Miles Brundage, Shahar Avin, Jack Clark, H.”
Explainable machine learning in deployment (2019, background): “Sincemachine learning models are “dual use” [12], we should be aware that in some settings, explanations or other tools could enable malicious users to increase capabilities and performance of undesirable systems.”
Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning (2019, background): “It is clear that advances in ML have the potential to be misused: the main example we have discussed here is how advances in synthetic media creation may be used to sow disinformation and mistrust (but many others can be, and have been discussed [2]).”
Regulating ChatGPT and other Large Generative AI Models (2023, background): “Generative models, as virtually any novel technology, may be used for better (think: birthday cards) or worse purposes ( think: shitstorm) [119].”
The tension between openness and prudence in AI research (2019, background): “However, concerns about potential harms arising from the misuse of advances in AI research are growing (Brundage et al., 2018; Ovadya and Whittlestone, 2019).”
Ethics of artificial intelligence and robotics (2020, background): “…it: It often talks as if we already knew what would be ethical, and the issues are just what future technology will bring, and what we should do about it; as if the issues were only considerations of risk, security (Brundage et al. 2018), and the prediction of impact (e.g. in the job market).”
Machines as teammates: A research agenda on AI in team collaboration (2020, background): “AI research has not yet produced technology capable of critical thinking and problem solving on par with human abilities, but progress is being made toward those goals [2].”
An Introduction to Deep Reinforcement Learning (2018, positive): “In addition, as with all powerful tools, deep RL algorithms also bring societal and ethical challenges (Brundage et al., 2018), raising the question of how they can be used for the benefit of all.”
Towards Automated Circuit Discovery for Mechanistic Interpretability (2023, background): “A”
The Fallacy of AI Functionality (2022, background): “The fear of hyper-competent AI systems also drives discussions on potentialmisuse [29].”
Pretrained AI Models: Performativity, Mobility, and Change (2019, neutral): “[11].”