Aligning AI Optimization to Community Well-Being

By Jonathan Stray. 2020

71 citations recorded by the source corpus.

This paper investigates incorporating community well-being metrics into the objectives of optimization algorithms and the teams that build them. It documents two cases where a large platform appears to have modified their system to this end. Facebook incorporated “well-being” metrics in 2017, while YouTube began integrating “user satisfaction” metrics around 2015. Metrics tied to community well-being outcomes could also be used in many other systems, such as a news recommendation system that tries to increase exposure to diverse views, or a product recommendation system that opstimizes for the carbon footprint of purchased products. Generalizing from these examples and incorporating insights from participatory design and AI governance leads to a proposed process for integrating community well-being into commercial AI systems: identify and involve the affected community, choose a useful metric, use this metric as a managerial performance measure and/or an algorithmic objective, and evaluate and adapt to outcomes. Important open questions include the best approach to community participation and the uncertain business effects of this process.

Subfields: Artificial Intelligence, Management Information Systems, Sociology and Political Science, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Information Systems

In Sociology and Political Science

In Safety, Robustness & Interpretability

In AI Ethics, Fairness & Society

Citations in the corpus

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (2022, background): “Optimizing for misspeciﬁed proxies led YouTube to overemphasize watch-time and harm user satisfaction (Stray, 2020), as well as to recommended extreme political content to users (Ribeiro et al., 2020).”
Unsolved Problems in ML Safety (2021, background): “Measurements such as clicks and watch time may be easily measurable, but they often leave out and work against important human values such as wellbeing [105, 52, 170, 171].”
An Overview of Catastrophic AI Risks (2023, background): “After all, if they stay engaged, surely that means they are getting some value from the content shown to them? However, in trying to maximize the time users spend on a platform, these systems often select enraging, exaggerated, and addictive content [104, 105].”
Large Language Model Alignment: A Survey (2023, background): “If an LLM-based agent finds a goal shortcut (Stray, 2020; Stray et al., 2021) or it is no longer “satisfied” with being controlled by humans under the drive of self-awareness, risky behaviors would emerge immediately.”
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society (2023, background): “AI alignment is a ﬁeld that aims to ensure that AI systems adhere to their intended goals, interests, and values, as envisioned by their designers [4, 39, 110, 32, 38, 74, 10].”
CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society (2023, background): “AI alignment is a field that aims to ensure that AI systems adhere to their intended goals, interests, and values, as envisioned by their designers [2, 25, 63, 20, 24, 43, 7].”