λOpt: Learn to Regularize Recommender Models in Finer Levels

By Yihong Chen, Bei Chen, Xiangnan He, Chen Gao, Yong Li, Jian-Guang Lou, Yue Wang. 2019

47 citations recorded by the source corpus.

Recommendation models mainly deal with categorical variables, such as user/item ID and attributes. Besides the high-cardinality issue, the interactions among such categorical variables are usually long-tailed, with the head made up of highly frequent values and a long tail of rare ones. This phenomenon results in the data sparsity issue, making it essential to regularize the models to ensure generalization. The common practice is to employ grid search to manually tune regularization hyperparameters based on the validation data. However, it requires non-trivial efforts and large computation resources to search the whole candidate space; even so, it may not lead to the optimal choice, for which different parameters should have different regularization strengths. In this paper, we propose a hyperparameter optimization method, lambdaOpt, which automatically and adaptively enforces regularization during training. Specifically, it updates the regularization coefficients based on the performance of validation data. With lambdaOpt, the notorious tuning of regularization hyperparameters can be avoided; more importantly, it allows fine-grained regularization (i.e. each parameter can have an individualized regularization coefficient), leading to better generalized models. We show how to employ lambdaOpt on matrix factorization, a classical model that is representative of a large family of recommender models. Extensive experiments on two public benchmarks demonstrate the superiority of our method in boosting the performance of top-K recommendation.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Liqiang Nie
5. Fuli Feng
6. Meng Wang
7. Wenjie Wang
8. Yong Li

In Management Science and Operations Research

1. Xiangnan He
2. Xiang Wang
3. Tat-seng Chua
4. Meng Wang
5. Fuli Feng
6. Liqiang Nie
7. Wenjie Wang
8. Yongdong Zhang

In Signal Processing

1. Liqiang Nie
2. Tat-seng Chua
3. Yongdong Zhang
4. Fuli Feng
5. Meng Wang
6. Xiangnan He
7. Weiwei Jiang

Citations in the corpus

• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation (2020, background): “ent. This is probably because the training data does not contain sufficient signal to learn goodαthat can generalize to unknown data. We have also tried to learn αfrom validation data, as inspired by [5] that learns hyper-parameters on validation data. The performance is slightly improved (less than”
• Denoising Implicit Feedback for Recommendation (2020, background): “For instance, a purchase is false-positive if the following rating score ([1, 5]) < 3.”