Harms from Increasingly Agentic Algorithmic Systems

By Alan Chan, Rebecca Salganik, A. Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco di Langosco, Zhonghao He, Yawen Duan, Micah Carroll and 12 more. 2023

259 citations recorded by the source corpus.

Research in Fairness, Accountability, Transparency, and Ethics (FATE)1 has established many sources and forms of algorithmic harm, in domains as diverse as health care, finance, policing, and recommendations. Much work remains to be done to mitigate the serious harms of these systems, particularly those disproportionately affecting marginalized communities. Despite these ongoing harms, new systems are being developed and deployed, typically without strong regulatory barriers, threatening the perpetuation of the same harms and the creation of novel ones. In response, the FATE community has emphasized the importance of anticipating harms, rather than just responding to them. Anticipation of harms is especially important given the rapid pace of developments in machine learning (ML). Our work focuses on the anticipation of harms from increasingly agentic systems. Rather than providing a definition of agency as a binary property, we identify 4 key characteristics which, particularly in combination, tend to increase the agency of a given algorithmic system: underspecification, directness of impact, goal-directedness, and long-term planning. We also discuss important harms which arise from increasing agency – notably, these include systemic and/or long-range impacts, often on marginalized or unconsidered stakeholders. We emphasize that recognizing agency of algorithmic systems does not absolve or shift the human responsibility for algorithmic harms. Rather, we use the term agency to highlight the increasingly evident fact that ML systems are not fully under human control. Our work explores increasingly agentic algorithmic systems in three parts. First, we explain the notion of an increase in agency for algorithmic systems in the context of diverse perspectives on agency across disciplines. Second, we argue for the need to anticipate harms from increasingly agentic systems. Third, we discuss important harms from increasingly agentic systems and ways forward for addressing them. We conclude by reflecting on implications of our work for anticipating algorithmic harms from emerging systems.

Subfields: Cognitive Neuroscience, Information Systems, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Information Systems

In Safety Research

In Safety, Robustness & Interpretability

In AI Ethics, Fairness & Society

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “They include but are not limited to generative models such as text-based language models (BLOOM [30], GPT-3 [40], OPT[271]), text-to-image models (ImaGen [205], DALL · E[170], Stable Diffusion [200]), and increasingly multimodal models [53] ( GPT-4 GPT-4 [ 171 ]).”
Mirages: On Anthropomorphism in Dialogue Systems (2023, background): “Rather than a single factor which makes humans human , Scruton (2017, (p. 31)) argues that humanity is emergent: each individual element does not make a human but collectively they make up the language of humanness.”
Multi-Agent Collaboration Mechanisms: A Survey of LLMs (2025, unknown): “These criteria include assessing the overall system performance [155], such as reasoning capabilities, task completion rates, as well as specific system characteristics like coordination efficiency and contextual appropriateness [17].”
Managing extreme AI risks amid rapid progress (2023, background): “Companies, governments, and militaries might be forced to deploy AI systems widely and cut back on expensive human verification of AI decisions, or risk being outcompeted 19,38 .”
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (2023, background): “There is an additional possibility that powerful AI systems will be highly profitable and serve to concentrate large amounts of wealth and power into the hands of a few [O’Keefe et al., 2020, Chan et al., 2023b].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025, unknown): “Without the ability to represent and reason about these causal links, Agentic AI systems are inherently constrained in their ability to safely generalize beyond narrow, predefined training scenarios [186,215].”
Agentic AI: Autonomous Intelligence for Complex Goals—A Comprehensive Survey (2025, unknown): “…the situation on the exchange, independently making decisions regarding the purchase or sale of securities, and adjusting their strategies to changing conditions in real time, which significantly improves the quality of managerial performance and reduces the participation of human beings [1].”