Deep Reinforcement Learning for List-wise Recommendations
By Xiangyu Zhao, L. Zhang, Zhuoye Ding, Dawei Yin, Yihong Eric Zhao, Jiliang Tang. 2017
195 citations recorded by the source corpus.
Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedbacks. In particular, we introduce an online user-agent interacting environment simulator, which can pre-train and evaluate model parameters offline before applying the model online. Moreover, we validate the importance of list-wise recommendations during the interactions between users and agent, and develop a novel approach to incorporate them into the proposed framework LIRD for list-wide recommendations. The experimental results based on a real-world e-commerce dataset demonstrate the effectiveness of the proposed framework.
Subfields: Information Systems, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Systems
- Xiangnan He
- Xiang Wang
- Fuli Feng
- Meng Wang
- Dawei Yin
- M. M. Afsar
- Trafford Crump
- Behrouz Homayoun Far
In Management Science and Operations Research
- Xiangnan He
- M. M. Afsar
- Trafford Crump
- Behrouz Homayoun Far
- Xiang Wang
- Meng Wang
- Fuli Feng
- Xiangyu Zhao
Citations in the corpus
- Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning (2018, background): “[33, 34] propose a novel page-wise recommendation framework based on deep reinforcement learning, which can optimize a page of”
- Deep reinforcement learning for page-wise recommendations (2018, methodology): “[43] optimized a set of recommendations with proper order.”
- Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, background): “Commonly it has applications in recommendation systems (Sunehag et al., 2015; Zhao et al., 2017; 2018; Ie et al., 2019b; Gong et al., 2019; Liu et al., 2021; Jiang et al., 2018; Song et al., 2020).”
- Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “Reinforcement learning [13], [144], [145], [146], [147], [148], [149], [150], [151], [152], [153], [154], [155], [156], [156], [157], [158], [159], [160] Others [161], [162]”
- Reinforcement Learning based Recommender Systems: A Survey (2021, methodology): “DRL-based Methods RLRS Year SR PO RF EB Metrics Dataset Application Code Slate-MDP [112] 2015 SR1 DQN R1 Simulation Reward N/A Wolpertinger [24] 2015 SR1 DDPG R1 Simulation Reward N/A Nemati et al. [113] 2016 SR2 DQN R2 Oline Reward MIMIC Healthcare CEI [114] 2017 SR3 REINFORCE-wb R1 Simulation P, F”
- A Survey on Session-based Recommender Systems (2019, methodology): “Model different types of dependencies, e.g., long and short term dependencies Relatively complex and costly [108],[132] Gener-ative Dynamic, incremental sessions Close to the practical session formation Complex [130],[135],[150] RL Dynamic, incremental sessions Interactive process, consider the futu”