Between Subjectivity and Imposition

By Milagros Miceli, Martin Schuessler, Tianling Yang. 2020

129 citations recorded by the source corpus.

The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators? subjectivity as a major cause of biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “More generally, several researchers have identified a widespread failure to recognize annotation work as interpretive work, which in turn can result in a conflation of target labels in a collected dataset and real-world objects, for which there may be no single ground truth label.(55,56) As discusse”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, background): “For example, publications accompanying new datasets under-specify the decisions that go into collection, curation, and annotation (Geiger et al., 2020; Scheuerman et al., 2020), and data labor often goes undocumented (Irani and Silberman, 2013; Gray and Suri, 2019; Miceli et al., 2020).”