Pretrained AI Models: Performativity, Mobility, and Change
By L. Varshney, Nitish Shirish Keskar, Richard Socher. 2019
25 citations recorded by the source corpus.
The paradigm of pretrained deep learning models has recently emerged in artificial intelligence practice, allowing deployment in numerous societal settings with limited computational resources, but also embedding biases and enabling unintended negative uses. In this paper, we treat pretrained models as objects of study and discuss the ethical impacts of their sociological position. We discuss how pretrained models are developed and compared under the common task framework, but that this may make self-regulation inadequate. Further how pretrained models may have a performative effect on society that exacerbates biases. We then discuss how pretrained models move through actor networks as a kind of computationally immutable mobile, but that users also act as agents of technological change by reinterpreting them via fine-tuning and transfer. We further discuss how users may use pretrained models in malicious ways, drawing a novel connection between the responsible innovation and user-centered innovation literatures. We close by discussing how this sociological understanding of pretrained models can inform AI governance frameworks for fairness, accountability, and transparency.
Subfields: Artificial Intelligence, AI Ethics, Fairness & Society, Social Sciences & Humanities
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Caiming Xiong
- Richard Socher
- Nitish Shirish Keskar
- Bryan McCann
- Mohit Bansal
- Nazneen Rajani
- Yejin Choi
- Luke S. Zettlemoyer
In AI Ethics, Fairness & Society
Citations in the corpus
- CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “Our actions reflect principles from a recent sociology-based AI governance framework that aims to expand responsible innovation to consider networks of users, dynamics, and feedback (Varshney et al., 2019).”
- BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, background): “Many of the safety and governance considerations for pretrained language models for human language also apply for protein language [78], e.”
- Limits of Detecting Text Generated by Large-Scale Language Models (2020, background): “Within the common task framework [11], there are leaderboards(1) that assess the perplexity of language models over standard datasets such as WikiText-103 [24].”