On Sampling Strategies for Neural Network-based Collaborative Filtering

By Ting-Li Chen, Yizhou Sun, Yue Shi, Liangjie Hong. 2017

258 citations recorded by the source corpus.

Recent advances in neural networks have inspired people to design hybrid recommendation algorithms that can incorporate both (1) user-item interaction information and (2) content information including image, audio, and text. Despite their promising results, neural network-based recommendation algorithms pose extensive computational costs, making it challenging to scale and improve upon. In this paper, we propose a general neural network-based recommendation framework, which subsumes several existing state-of-the-art recommendation algorithms, and address the efficiency issue by investigating sampling strategies in the stochastic gradient descent training for the framework. We tackle this issue by first establishing a connection between the loss functions and the user-item interaction bipartite graph, where the loss function terms are defined on links while major computation burdens are located at nodes. We call this type of loss functions "graph-based" loss functions, for which varied mini-batch sampling strategies can have different computational costs. Based on the insight, three novel sampling strategies are proposed, which can significantly improve the training efficiency of the proposed framework (up to $\times 30$ times speedup in our experiments), as well as improving the recommendation performance. Theoretical analysis is also provided for both the computational cost and the convergence. We believe the study of sampling strategies have further implications on general graph-based loss functions, and would also enable more research under the neural network-based recommendation framework.

Subfields: Artificial Intelligence, Information Systems, Optimization & Training, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Optimization & Training

In Recommender Systems

Citations in the corpus

Unsupervised Dense Information Retrieval with Contrastive Learning (2021, methodology): “This method has been widely used to train information retrieval models with supervised data Chen et al. (2017b); Karpukhin et al. (2020) and was also considered when using ICT to pre-train retrievers by Lee et al. (2019).”
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, background): “Explaining away is known in reasoning as the concept that the conﬁrmation of one cause of an observed event dismisses alternative causes (Pearl, 1988; Wellman & Henrion, 1993).”
SimCSE: Simple Contrastive Learning of Sentence Embeddings (2021, methodology): “In this work we follow the contrastive framework in Chen [35] and take a cross-entropy objective with in-batch negatives [36, 37].”
Text Is All You Need: Learning Language Representations for Sequential Recommendation (2023, methodology): “In-batch negative instances [3] are using ground truth items of other instance sequences in the same batch as negative items.”
Graph Contrastive Learning with Augmentations (2020, background): “Negative pairs are not explicitly sampled but generated from the other N − 1 augmented graphs within the same minibatch as in [53, 18].”
Joint Text Embedding for Personalized Content-based Recommendation (2017, methodology): “vised text embedding techniques [16, 18, 19], which do not require labels but cannot adapt to the task of interest. We further generalize the proposed model and develop e†cient training techniques in [3]. 6 CONCLUSIONS In this work, we tackle the problem of content-based recommendation for completel”
SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, background): “Negative pairs are generated from the other N − 1 perturbed representations within the same mini-batch as in [5, 42, 56].”
A Survey of Deep Learning for Scientific Discovery (2020, background): “Many of these mutual information based metrics also rely on contrastive losses [30], which, at a high level, provides supervision to the network by making representations of a pair of similar inputs more similar than representations of a pair of diﬀerent inputs.”