The Effect of Population and "Structural" Biases on Social Media-based Algorithms: A Case Study in Geolocation Inference Across the Urban-Rural Spectrum
By Isaac L. Johnson, Connor McMahon, Johannes Schöning, Brent J. Hecht. 2017
66 citations recorded by the source corpus.
Much research has shown that social media platforms have substantial population biases. However, very little is known about how these population biases affect the many algorithms that rely on social media data. Focusing on the case study of geolocation inference algorithms and their performance across the urban-rural spectrum, we establish that these algorithms exhibit significantly worse performance for underrepresented populations (i.e. rural users). We further establish that this finding is robust across both text- and network-based algorithm designs. However, we also show that some of this bias can be attributed to the design of algorithms themselves rather than population biases in the underlying data sources. For instance, in some cases, algorithms perform badly for rural users even when we substantially overcorrect for population biases by training exclusively on rural data. We discuss the implications of our findings for the design and study of social media-based algorithms.
Subfields: Statistical and Nonlinear Physics, Transportation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistical and Nonlinear Physics
- Alexandra Olteanu
- Fernando Diaz
- Carlos Castillo
- Emre Kıcıman
- Yulia Tsvetkov
- Dan Jurafsky
- David Jurgens
- Muhammad Imran
In Transportation
- Alexandra Olteanu
- Carlos Castillo
- Fernando Diaz
- David Jurgens
- Emre Kıcıman
- Olga Russakovsky
- Angelina Wang
- Ricardo A. Baeza-Yates
Citations in the corpus
- Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “They can also impact the performance of algorithms that make inferences about users (Johnson et al., 2017), further compromising the internal validity of both type I and type II research.”
- Incorporating Dialectal Variability for Socially Equitable Language Identification (2017, methodology): “The method is accurate to within tens of kilometers on urban and rural users (Johnson et al., 2017), which is sufficient for the city-level analysis we use here.”
- A Qualitative Exploration of Perceptions of Algorithmic Fairness (2018, background): “…urban, white, non-Hispanic populations, for example, potentially attracting more tourist commerce to their neighborhoods [15], and Johnson et al. found that geolocation inference algorithms exhibited substantially worse performance for underrepresented populations, i.e., rural users [47].”