Calibrated recommendations

By Harald Steck. ACM Conference on Recommender Systems, 2018

331 citations recorded by the source corpus.

When a user has watched, say, 70 romance movies and 30 action movies, then it is reasonable to expect the personalized list of recommended movies to be comprised of about 70% romance and 30% action movies as well. This important property is known as calibration, and recently received renewed attention in the context of fairness in machine learning. In the recommended list of items, calibration ensures that the various (past) areas of interest of a user are reflected with their corresponding proportions. Calibration is especially important in light of the fact that recommender systems optimized toward accuracy (e.g., ranking metrics) in the usual offline-setting can easily lead to recommendations where the lesser interests of a user get crowded out by the user's main interests-which we show empirically as well as in thought-experiments. This can be prevented by calibrated recommendations. To this end, we outline metrics for quantifying the degree of calibration, as well as a simple yet effective re-ranking algorithm for post-processing the output of recommender systems.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/5b156a3749b4110e328bcf2b147dc3d71ac09fc7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Fuli Feng
3. Yiqun Liu
4. Shaoping Ma
5. Rohan Taori
6. Yongdong Zhang
7. Tat-seng Chua
8. Tianxin Wei

In Information Systems

1. Xiangnan He
2. Fuli Feng
3. Yiqun Liu
4. Shaoping Ma
5. Yongfeng Zhang
6. Yongdong Zhang
7. Tianxin Wei
8. Tat-seng Chua

Citations in the corpus

• Fairness in Machine Learning: A Survey (2020, methodology): “Subsequent publications expanded this view of recommender fairness by proposing additional metrics as well as algorithms [281, 294, 254, 29, 58].”
• Causal Intervention for Leveraging Popularity Bias in Recommendation (2021, background): “Worse still, the feedback loop ecology of recommender system further intensifies such Matthew effect [29], causing notorious issues like echo chamber [16] and filter bubble [41].”
• A Survey on the Fairness of Recommender Systems (2022, methodology): “Divided by Optimization Object Treatment [20, 29, 30, 33, 36, 48, 55, 60, 76, 83, 93, 95, 97, 103, 115] [19, 28, 31, 45, 58, 59, 62, 67, 71, 82, 86, 90, 107, 112, 116] [7, 8, 12, 15, 23, 44, 52, 64, 75, 81, 84, 85, 87, 94, 114] [61, 66, 77, 80, 99, 102, 106] Impact [6, 19, 22, 25, 33, 39, 43, 53, 54”
• A Systematic Study of Bias Amplification (2022, background): “This is especially true in systems that use model prediction scores (rather than hard labels), such as when building ranking and recommendation systems [40].”
• Data Feedback Loops: Model-driven Amplification of Dataset Biases (2022, background): “This issue arises due to the tension between improving ranking metrics and considerations of bias, fairness, or diversity (Steck, 2018; Chaney et al., 2018).”