Exploring Stereotypes and Biased Data with the Crowd

By Zeyuan Hu, Julia Strout. 2018

5 citations recorded by the source corpus.

The goal of our research is to contribute information about how useful the crowd is at anticipating stereotypes that may be biasing a data set without a researcher's knowledge. The results of the crowd's prediction can potentially be used during data collection to help prevent the suspected stereotypes from introducing bias to the dataset. We conduct our research by asking the crowd on Amazon's Mechanical Turk (AMT) to complete two similar Human Intelligence Tasks (HITs) by suggesting stereotypes relating to their personal experience. Our analysis of these responses focuses on determining the level of diversity in the workers' suggestions and their demographics. Through this process we begin a discussion on how useful the crowd can be in tackling this difficult problem within machine learning data collection.

Subfields: Computer Science Applications, Modeling and Simulation, Ocean Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

1. Amit K. Roy-Chowdhury
2. Rameswar Panda
3. Niluthpol Chowdhury Mithun

Citations in the corpus

• Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “Moreover, it is generally feasible to have only a limited number of users to annotate training images, which may lead to a biased model [17, 50, 59].”