Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems

By Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, Fernando Diaz. 2018

323 citations recorded by the source corpus.

Two-sided marketplaces are platforms that have customers not only on the demand side (e.g. users), but also on the supply side (e.g. retailer, artists). While traditional recommender systems focused specifically towards increasing consumer satisfaction by providing relevant content to consumers, two-sided marketplaces face the problem of additionally optimizing for supplier preferences, and visibility. Indeed, the suppliers would want afair opportunity to be presented to users. Blindly optimizing for consumer relevance may have a detrimental impact on supplier fairness. Motivated by this problem, we focus on the trade-off between objectives of consumers and suppliers in the case of music streaming services, and consider the trade-off betweenrelevance of recommendations to the consumer (i.e. user) andfairness of representation of suppliers (i.e. artists) and measure their impact on consumersatisfaction. We propose a conceptual and computational framework using counterfactual estimation techniques to understand, and evaluate different recommendation policies, specifically around the trade-off between relevance and fairness, without the need for running many costly A/B tests. We propose a number of recommendation policies which jointly optimize relevance and fairness, thereby achieving substantial improvement in supplier fairness without noticeable decline in user satisfaction. Additionally, we consider user disposition towards fair content, and propose a personalized recommendation policy which takes into account consumer's tolerance towards fair content. Our findings could guide the design of algorithms powering two-sided marketplaces, as well as guide future research on sophisticated algorithms for joint optimization of user relevance, satisfaction and fairness.

Subfields: Management of Technology and Innovation, Marketing, Strategy and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Marketing

In Strategy and Management

Citations in the corpus

Manipulating and Measuring Model Interpretability (2018, background): “Machine learning models also infuence people’s decisions about what news articles to read [3, 13, 69, 88, 104], what movies to watch [10], what music to listen to [77], what clothes to buy [20], and even who to date [90].”
User-oriented Fairness in Recommendation (2021, background): “[29] address the supplier fairness in two-sided marketplace platforms and propose a heuristic strategy to jointly optimize fairness and performance.”
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “Specifically, according to the involved stakeholders, fairness in recommender systems can be divided into two categories [4, 98, 128]: user fairness, which attempts to ensure no algorithmic bias among specific users or demographic groups [9, 90, 96], and item fairness, which indicates fair exposures”
Fairness in Recommendation Ranking through Pairwise Comparisons (2019, methodology): “In this paper we focus on the risk of a recommender system under-ranking groups of items [21, 8, 39].”