Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes
By David Gray Widder, Dawn Nafus, Laura A. Dabbish, James D. Herbsleb. 2022
75 citations recorded by the source corpus.
Open source software communities are a significant site of AI development, but “Ethical AI” discourses largely focus on the problems that arise in software produced by private companies. Design, policy and tooling interventions to encourage “Ethical AI” based on studies in private companies risk being ill-suited for an open source context, which operates under radically different organizational structures, cultural norms, and incentives. In this paper, we show that significant and understudied harms and possibilities originate from differing practices of transparency and accountability in the open source community. We conducted an interview study of an AI-enabled open source Deepfake project to understand how members of that community reason about the ethics of their work. We found that notions of the “Freedom 0” to use code without any restriction, alongside beliefs about technology neutrality and technological inevitability, were central to how community members framed their responsibilities, and the actions they believed were and were not available to them. We propose a continuum between harms resulting from how a system is implemented versus how it is used, and show how commitments to radical transparency in open source allow great ethical scrutiny for harms wrought by implementation bugs, but allow harms through (mis)use to proliferate, requiring a deeper toolbox for disincentivizing harmful use. We discuss how an assumption of control over downstream uses is often implicit in discourses of “Ethical AI”, but outline alternative possibilities for action in cases such as open source where this assumption may not hold.
Subfields: Computer Science Applications, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Science Applications
In Safety Research
In Safety, Robustness & Interpretability
In AI Ethics, Fairness & Society
Citations in the corpus
- The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Examining the specific use case of open-source Deepfakes highlights the difficulty of managing downstream harms in real-time and the risk of safety controls being seen as futile [103].”