A General Framework for Counterfactual Learning-to-Rank

By Aman Agarwal, Kenta Takatsu, Ivan Zaitsev, Thorsten Joachims. 2018

149 citations recorded by the source corpus.

Implicit feedback (e.g., click, dwell time) is an attractive source of training data for Learning-to-Rank, but its naive use leads to learning results that are distorted by presentation bias. For the special case of optimizing average rank for linear ranking functions, however, the recently developed SVM-PropRank method has shown that counterfactual inference techniques can be used to provably overcome the distorting effect of presentation bias. Going beyond this special case, this paper provides a general and theoretically rigorous framework for counterfactual learning-to-rank that enables unbiased training for a broad class of additive ranking metrics (e.g., Discounted Cumulative Gain (DCG)) as well as a broad class of models (e.g., deep networks). Specifically, we derive a relaxation for propensity-weighted rank-based metrics which is subdifferentiable and thus suitable for gradient-based optimization. We demonstrate the effectiveness of this general approach by instantiating two new learning methods. One is a new type of unbiased SVM that optimizes DCG - called SVM PropDCG - and we show how the resulting optimization problem can be solved via the Convex Concave Procedure (CCP). The other is Deep PropDCG, where the ranking function can be an arbitrary deep network. In addition to the theoretical support, we empirically find that SVM PropDCG significantly outperforms existing linear rankers in terms of DCG. Moreover, the ability to train non-linear ranking functions via Deep PropDCG further improves performance.

Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Jimmy J. Lin
4. Fuli Feng
5. Andrew Yates
6. Rodrigo Nogueira
7. Meng Wang
8. Tat-seng Chua

In Computer Networks and Communications

1. Xiang Wang
2. Rodrigo Nogueira
3. Andrew Yates
4. Meng Wang
5. Jiawei Chen
6. Jimmy J. Lin
7. Fuli Feng
8. Jinfeng Yi

In Management Science and Operations Research

1. Xiangnan He
2. Xiang Wang
3. Meng Wang
4. Rodrigo Nogueira
5. Fuli Feng
6. Jimmy J. Lin
7. Andrew Yates
8. Jiawei Chen

In Information Retrieval & Search

1. Jimmy J. Lin
2. Rodrigo Nogueira
3. Andrew Yates
4. Zeynep Akkalyoncu Yilmaz
5. Xiang Wang
6. Fuli Feng
7. Jinfeng Yi
8. Meng Wang

Citations in the corpus

• Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “Although a specific thread of work in the learning to rank tradition, called “counterfactual learning to rank” [Agarwal et al., 2019] remains active to this day.”
• Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System (2020, background): “The main purpose of introducing causal inference in recommender systems is to remove the bias [4, 5, 8, 25, 42, 48, 57].”
• Bias and Debias in Recommender System: A Survey and Future Directions (2020, methodology): “Another common solution to correct position bias is employing inverse propensity score, where each instance is weighted with a position-aware values [93].”