Sampling-bias-corrected neural modeling for large corpus item recommendations

By Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, Ed H. Chi. ACM Conference on Recommender Systems, 2019

344 citations recorded by the source corpus.

Many recommendation systems retrieve and score items from a very large corpus. A common recipe to handle data sparsity and power-law item distribution is to learn item representations from its content features. Apart from many content-aware systems based on matrix factorization, we consider a modeling framework using two-tower neural net, with one of the towers (item tower) encoding a wide variety of item content features. A general recipe of training such two-tower models is to optimize loss functions calculated from in-batch negatives, which are items sampled from a random mini-batch. However, in-batch loss is subject to sampling biases, potentially hurting model performance, particularly in the case of highly skewed distribution. In this paper, we present a novel algorithm for estimating item frequency from streaming data. Through theoretical analysis and simulation, we show that the proposed algorithm can work without requiring fixed item vocabulary, and is capable of producing unbiased estimation and being adaptive to item distribution change. We then apply the sampling-bias-corrected modeling approach to build a large scale neural retrieval system for YouTube recommendations. The system is deployed to retrieve personalized suggestions from a corpus with tens of millions of videos. We demonstrate the effectiveness of sampling-bias correction through offline experiments on two real-world datasets. We also conduct live A/B testings to show that the neural retrieval system leads to improved recommendation quality for YouTube.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/41a08dd12ab4495a8f617174833302484dcce055

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Self-supervised Learning for Large-scale Item Recommendations (2020, methodology): “We partitioned the dataset into training and evaluation using a (90%, 10%) split, following the same treatment in [23] and [39].”
Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited (2023, background): “This is reasonable since IDRec is known to to be good at modeling popular items according to the existing literature [5, 74, 76].”
Long-tail learning via logit adjustment (2020, background): “Interestingly, for τ = − 1 , a similar loss to (10) has been considered in the context of negative sampling for scalability [Yi et al., 2019]: here, one samples a small subset of negatives based on the class priors π , and applies logit correction to obtain an unbiased estimate of the unsampled…”
Learning to Embed Categorical Features without Embedding Tables for Recommendation (2020, methodology): “[30, 34, 36]), and widely used in production-scale systems for handling large-vocab and out-of-vocab categorical features (e.g. Youtube [37] and Twitter [38]).”
The Benchmark Lottery (2021, background): “( i ) Different recommendation platforms tackle slightly different problems (e.g retrieval [Yi et al., 2019], ranking ([Pei et al., 2019]), or multitask learning ([Zhao et al., 2019])), and each requires their own evaluation setup.”
Recommender Systems with Generative Retrieval (2023, background): “Not only does this require a huge memory and storage footprint, but it also leads to imbalance when training the embeddings of the items, where popular items will be over-sampled as compared to infrequent ones [41].”