The Values Encoded in Machine Learning Research

By Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, Michelle Bao. 2021

401 citations recorded by the source corpus.

Machine learning currently exerts an outsized influence on the world, increasingly affecting institutional practices and impacted communities. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we first introduce a method and annotation scheme for studying the values encoded in documents such as research papers. Applying the scheme, we analyze 100 highly cited machine learning papers published at premier machine learning conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: their justification for their choice of project, which attributes of their project they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that few of the papers justify how their project connects to a societal need (15%) and far fewer discuss negative potential (1%). Through line-by-line content analysis, we identify 59 values that are uplifted in ML research, and, of these, we find that the papers most frequently justify and assess themselves based on Performance, Generalization, Quantitative evidence, Efficiency, Building on past work, and Novelty. We present extensive textual evidence and identify key themes in the definitions and operationalization of these values. Notably, we find systematic textual evidence that these top values are being defined and applied with assumptions and implications generally supporting the centralization of power. Finally, we find increasingly close ties between these highly cited papers and tech companies and elite universities.

Subfields: Artificial Intelligence, Information Systems, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, neutral): “When the development efforts are guided by prioritizing internal definitions of performance over their impact on society, the values of the developers come to be emphasized over those of the direct and indirect users (Birhane et al., 2022).”
Measuring the Carbon Intensity of AI in Cloud Instances (2022, background): “When considering the net impacts of an AI application, it is imperative to determine the extent to which AI is incentivizing practices that have a negative impact on the environment, or the extent to which applications are directly reducing emissions or otherwise incentivizing practices that are ben”
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, methodology): “Following the advice of Birhane et al. (2021), in this section we seek to explicitly document our choices, rationales, and values in designing and implementing Pythia .”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “The many technical and social aspects of AI systems [15] require robust research [75] conducted with communities affected [11] to ensure these systems benefit and do not exploit marginalized groups, if the systems are to be deployed among these groups.”
BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model (2022, background): “The beneﬁts of such an inclusive and diverse participatory approach to research, what Birhane et al. (2022) call the “partic-ipatory turn” of AI research, goes well beyond the Big Science metaphor and is indeed well aligned with trends observed by ?”
Multimodal datasets: misogyny, pornography, and malignant stereotypes (2021, background): “The culture in machine learning is such that ideas that promise improvements in training speed, model-size or top-k accuracy improvements are rapidly embraced while ideas and revelations pertaining to unethical aspects of datasets are either ignored or take a long time to lead to changes [62].”
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset (2023, methodology): “, 2021) and value-driven (Birhane et al., 2021) endeavor.”
Unsolved Problems in ML Safety (2021, background): “Beyond technical work, policy and governance work will be integral to safe deployment [43, 12, 16, 210, 25].”
Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022, background): “[18] review recent literature in ML to identify values that are typically put forward to motivate work in that field.”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “To tackle this issue, ML engineers should explicitly state the underlying systemic values, as systems will inevitably comprise certain assumptions (Birhane et al., 2021).”
Power to the People? Opportunities and Challenges for Participatory AI (2022, background): “The core motivation and objectives of this form of participation are somewhat incompatible with the core values of AI research, which include eﬃciency, performance, accuracy, novelty, and beating state-of-the-art [13].”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, background): “This reflects a deep value of the field [17], which prioritizes optimizing decision outcomes rather than the experience of human decision makers.”
Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, background): “Moreover, large datasets require computing resources, so projects can only be pursued by large entities at the possible expense of the individual and smaller research groups [8].”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, methodology): “The Test and Evaluation stage is continuous throughout the entire AI Development Lifecycle.”
Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “This focus is pervasive throughout machine learning scholarship and practice [8], perhaps best evidenced by the Common Task Framework [27], through which independent researchers compare predictive performance on common datasets.”
Evaluation Gaps in Machine Learning Practice (2022, background): “Birhane et al . find evidence for this in their study of the discourse of ML papers, showing that the field centers accuracy, generalization, and novelty, while marginalizing values such as safety [18].”