Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
By J. Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, K. G. Karahalios. 2017
236 citations recorded by the source corpus.
Search systems in online social media sites are frequently used to find information about ongoing events and people. For topics with multiple competing perspectives, such as political events or political candidates, bias in the top ranked results significantly shapes public opinion. However, bias does not emerge from an algorithm alone. It is important to distinguish between the bias that arises from the data that serves as the input to the ranking system and the bias that arises from the ranking system itself. In this paper, we propose a framework to quantify these distinct biases and apply this framework to politics-related queries on Twitter. We found that both the input data and the ranking system contribute significantly to produce varying amounts of bias in the search results and in different ways. We discuss the consequences of these biases and possible mechanisms to signal this bias in social media search systems' interfaces.
Subfields: Sociology and Political Science, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Sociology and Political Science
- Alexandra Olteanu
- Fernando Diaz
- Carlos Castillo
- Emre Kıcıman
- Isaac L. Johnson
- Fabiana Zollo
- Walter Quattrociocchi
- Inioluwa Deborah Raji
In Statistical and Nonlinear Physics
- Alexandra Olteanu
- Fernando Diaz
- Carlos Castillo
- Emre Kıcıman
- Fabiana Zollo
- Walter Quattrociocchi
- Antonio Scala
- Isaac L. Johnson
Citations in the corpus
- Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “For instance, Kulshrestha et al. (2017) introduces a framework for auditing search systems on social media platforms by differentiating between various sources of bias (e.g., due to content or due to ranking algorithms).”
- Addressing Age-Related Bias in Sentiment Analysis (2018, background): “Other researchers have attempted to systematically sort out user bias (e.g., keywords that are put in) from system bias [43].”
- The COVID-19 social media infodemic (2020, background): “f content and may amplify rumors and questionable information. Taking into account users’ preferences and attitudes, algorithms mediate and facilitate content promotion and thus information spreading [10]. This shift of paradigm profoundly impacts the construction of social perceptions [11] and the ”
- Discrimination through Optimization (2019, background): “Among most notable results, researchers have shown price discrimination in online retail sites [42], gender discrimination in job sites [16, 43], stereotypical gender roles re-enforced by online translation services [12] and image search [46], disparate performance on gender classi cation for Black ”
- Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products (2019, background): “Given these risks, much algorithmic audit work has focused on goals to gauge user awareness of algorithmic bias [17, 26] or evaluate the impact of bias on user behaviour and outcomes [16, 19, 24], instead of directly challenging companies to change commercial systems.”
- Fair Personalization (2017, background): “, it has been shown that ranking and classification algorithms can discriminate in online search results [4, 16] and allow for gender inequality in serving targeted advertising for high-paying jobs [6, 8, 23].”