Social Cues, Social Biases: Stereotypes in Annotations on People Images

By Jahna Otterbacher. 2018

12 citations recorded by the source corpus.

Human computation is often subject to systematic biases. We consider the case of linguistic biases and their consequences for the words that crowdworkers use to describe people images in an annotation task. Social psychologists explain that when describing oth- ers, the subconscious perpetuation of stereotypes is in- evitable, as we describe stereotype-congruent people and/or in-group members more abstractly than others. In an MTurk experiment we show evidence of these bi- ases, which are exacerbated when an image’s “popular tags” are displayed, a common feature used to provide social information to workers. Underscoring recent calls for a deeper examination of the role of training data quality in algorithmic biases, results suggest that it is rather easy to sway human judgment.

Subfields: Artificial Intelligence, Computer Science Applications, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. David A. Forsyth
2. Shruti Bhargava
3. Su Lin Blodgett
4. Morgan Klaus Scheuerman
5. Jahna Otterbacher
6. Solon Barocas
7. Jared Katzman
8. Angelina Wang

In Computer Science Applications

1. David A. Forsyth
2. Shruti Bhargava
3. Jahna Otterbacher

In Safety Research

1. David A. Forsyth
2. Shruti Bhargava
3. Morgan Klaus Scheuerman
4. Solon Barocas
5. Su Lin Blodgett
6. Angelina Wang
7. Adrian Weller

In Safety, Robustness & Interpretability

1. David A. Forsyth
2. Shruti Bhargava
3. Su Lin Blodgett
4. Morgan Klaus Scheuerman
5. Jahna Otterbacher
6. Solon Barocas
7. Angelina Wang
8. Yulia Tsvetkov

Citations in the corpus

• Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models (2019, background): “Annotators’ background and prior experiences tend to influence annotations [Otterbacher, 2018].”
• Taxonomizing and Measuring Representational Harms: A Look at Image Tagging (2023, background): “, similarly composed photos of CEOs of different genders), again testing for any differences in tagging behaviors (Otterbacher 2018; Steed and Caliskan 2021; Stock and Cisse 2018).”
• How Do We Talk about Other People? Group (Un)Fairness in Natural Language Image Descriptions (2019, background): “, (Otterbacher 2018)), previous work has considered social biases in datasets of images collected in the wild (e.”