Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search

By Sahin Cem Geyik, Stuart Ambler, Krishnaram Kenthapadi. Knowledge Discovery and Data Mining, 2019

451 citations recorded by the source corpus.

We present a framework for quantifying and mitigating algorithmic bias in mechanisms designed for ranking individuals, typically used as part of web-scale search and recommendation systems. We first propose complementary measures to quantify bias with respect to protected attributes such as gender and age. We then present algorithms for computing fairness-aware re-ranking of results. For a given search or recommendation task, our algorithms seek to achieve a desired distribution of top ranked results with respect to one or more protected attributes. We show that such a framework can be tailored to achieve fairness criteria such as equality of opportunity and demographic parity depending on the choice of the desired distribution. We evaluate the proposed algorithms via extensive simulations over different parameter choices, and study the effect of fairness-aware ranking on both bias and utility measures. We finally present the online A/B testing results from applying our framework towards representative ranking in LinkedIn Talent Search, and discuss the lessons learned in practice. Our approach resulted in tremendous improvement in the fairness metrics (nearly three fold increase in the number of search queries with representative results) without affecting the business metrics, which paved the way for deployment to 100% of LinkedIn Recruiter users worldwide. Ours is the first large-scale deployed framework for ensuring fairness in the hiring domain, with the potential positive impact for more than 630M LinkedIn members.

Subfields: Computer Science Applications, Management Science and Operations Research, Safety Research

View Paper: https://www.semanticscholar.org/paper/ef2c54044ab93189884829fa5ef47515e8da93a7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Management Science and Operations Research

In Safety Research

Citations in the corpus

Mitigating bias in algorithmic hiring: evaluating claims and practices (2019, methodology): “In the computer science literature, there are ongoing eﬀorts to deﬁne technical constraints on rankings in the spirit of equal representation and the 4 / 5 rule [18, 91, 94], and LinkedIn has adopted a similar approach to encourage demographic diversity in its search results [41].”
Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “user attributes, the concept of fairness has been generalized to multiple dimensions in recommender systems [178], spanning from fairness-aware ranking [114], [115], [116], supplier fairness in two-sided marketplace platforms [179], provider-side fairness to make items from different providers have ”
A Survey on the Fairness of Recommender Systems (2022, methodology): “For items, some researchers [31, 102] consider unfairness across different positions and then aggregate them.”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “Outside language modeling, the broader machine learning (ML) fairness community has documented sociotechnical 3 insights that can help bridge the gap between foresight and evaluation, drawing on domains including medical applications [80], facial recognition [22], and recommender systems [41].”
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, methodology): “Specifically, researchers adopt two methods to enhance fairness: One is directly debiasing recommendation results in the training process [9, 254], while the other endeavors to rank items to alleviate the unfairness in a post-processing method [55, 153].”
Gendered Language in Resumes and its Implications for Algorithmic Bias in Hiring (2022, background): “This calls for active consideration of fairness in downstream tasks such as employing “fairness through awareness” techniques (Dwork et al., 2012; Geyik et al., 2019; Zehlike et al., 2017) that explicitly take into account the protected class to achieve individual or group-level fairness.”