Unbiased Learning-to-Rank with Biased Feedback

By Thorsten Joachims, Adith Swaminathan, Tobias Schnabel. 2016

656 citations recorded by the source corpus.

Implicit feedback (e.g., clicks, dwell times, etc.) is an abundant source of data in human-interactive systems. While implicit feedback has many advantages (e.g., it is inexpensive to collect, user centric, and timely), its inherent biases are a key obstacle to its effective use. For example, position bias in search rankings strongly influences how many clicks a result receives, so that directly using click data as a training signal in Learning-to-Rank (LTR) methods yields sub-optimal results. To overcome this bias problem, we present a counterfactual inference framework that provides the theoretical basis for unbiased LTR via Empirical Risk Minimization despite biased data. Using this framework, we derive a Propensity-Weighted Ranking SVM for discriminative learning from implicit feedback, where click models take the role of the propensity estimator. In contrast to most conventional approaches to de-biasing the data using click models, this allows training of ranking functions even in settings where queries do not repeat. Beyond the theoretical support, we show empirically that the proposed learning method is highly effective in dealing with biases, that it is robust to noise and propensity model misspecification, and that it scales efficiently. We also demonstrate the real-world applicability of our approach on an operational search engine, where it substantially improves retrieval performance.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Noah A. Smith
4. Meng Wang
5. Jianfeng Gao
6. Chunhua Shen
7. Fuli Feng
8. Donald Metzler

In Computer Networks and Communications

1. Thorsten Joachims
2. Guansong Pang
3. Chunhua Shen
4. Xiang Wang
5. Xiangnan He
6. Meng Wang
7. Ed H. Chi
8. Fuli Feng

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Meng Wang
4. Fuli Feng
5. Yong Li
6. Chen Gao
7. Ed H. Chi
8. Thorsten Joachims

Citations in the corpus

• A General Framework for Counterfactual Learning-to-Rank (2018, methodology): “This is achieved by the following hinge-loss upper bound [15] on the rank”
• Fairness of Exposure in Rankings (2018, background): “ion bias to vj = 1 log(1+j) just like in the standard definition of DCG. More generally, one can also plug in the actual position-bias value, which can be estimated through an intervention experiment [18]. Job-seeker example. We come back to the job-seeker example from the introduction, and as illus”
• Disentangling User Interest and Conformity for Recommendation with Causal Embedding (2021, methodology): “• IPS [26, 40]: IPS eliminates popularity bias by re-weighting each instance according to item popularity.”
• Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback (2019, background): “As in the information retrieval field [11, 12, 17, 27], one should estimate the relevance of an item to a user from implicit feedback to improve user experience, because one can recommend items that users are genuinely interested in by accurately predicting its relevance.”
• Recommending what video to watch next: a multitask ranking system (2019, background): “Other approaches try to learn a bias term from position and apply it as a normalizer or regularizer [23, 34, 41].”
• Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting (2020, methodology): “For example, the Inverse Propensity Score (IPS) (Rosen-baum and Rubin, 1983) method has been successfully applied for causal effect analyses (Austin and Stuart, 2015), selection bias (Schonlau et al., 2009), position bias (Wang et al., 2018; Joachims et al., 2017) and so on.”
• Deep Learning for Anomaly Detection (2020, background): “osing a weak assumption of the ordinal order between anomaly and normal instances. (iii) This approach may build upon well-established ranking techniques and theories from areas like learning to rank [70,87,89]. Disadvantages. Their disadvantages are as follows. (i) At least some form of labeled ano”
• CAB: Continuous Adaptive Blending for Policy Evaluation and Learning (2019, methodology): “Finally, unlike the DR estimator, CAB can be used in off-policy Learning to Rank (LTR) algorithms like (Joachims et al., 2017), and CAB is specifically designed to control the bias/variance trade-off.”
• Fairness in Recommendation Ranking through Pairwise Comparisons (2019, background): “Two that relate to the challenges in fairness are the temporal dynamics [33, 48, 26, 9] and biased training data [29, 15, 3].”
• Top-K Off-Policy Correction for a REINFORCE Recommender System (2018, methodology): “Joachims et al. [21] learns a model of logged feedback for an unbiased ranking model; we take a similar perspective but use a DNN to model the logged behavior policy required for the off-policy learning.”
• Position Bias Estimation for Unbiased Learning to Rank in Personal Search (2018, methodology): “Among them, the position bias has a strong influence on users’ clicks [20].”
• Bias and Debias in Recommender System: A Survey and Future Directions (2020, methodology): “[94] demonstrate the ranking model trained with the IPS-based loss function will converge to the same model trained with true relevance labels.”