DRN: A Deep Reinforcement Learning Framework for News Recommendation

By Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, Zhenhui Jessie Li. 2018

748 citations recorded by the source corpus.

In this paper, we propose a novel Deep Reinforcement Learning framework for news recommendation. Online personalized news recommendation is a highly challenging problem due to the dynamic nature of news features and user preferences. Although some online recommendation models have been proposed to address the dynamic nature of news recommendation, these methods have three major issues. First, they only try to model current reward (e.g., Click Through Rate). Second, very few studies consider to use user feedback other than click / no click labels (e.g., how frequent user returns) to help improve recommendation. Third, these methods tend to keep recommending similar news to users, which may cause users to get bored. Therefore, to address the aforementioned challenges, we propose a Deep Q-Learning based recommendation framework, which can model future reward explicitly. We further consider user return pattern as a supplement to click / no click label in order to capture more user feedback information. In addition, an effective exploration strategy is incorporated to find new attractive news for users. Extensive experiments are conducted on the offline dataset and online production environment of a commercial news recommendation application and have shown the superior performance of our methods.

Subfields: Computer Vision and Pattern Recognition, Information Systems, Management Science and Operations Research, Reinforcement Learning, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiangnan He
2. Xiang Wang
3. Fuli Feng
4. Meng Wang
5. Yongfeng Zhang
6. Yi Tay
7. Dawei Yin
8. Kun Zhang

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Xing Xie
4. Meng Wang
5. Yongfeng Zhang
6. Fuli Feng
7. Fangzhao Wu
8. Chuhan Wu

In Management Science and Operations Research

1. Xiangnan He
2. Xiang Wang
3. Meng Wang
4. Fuli Feng
5. Kun Zhang
6. Xiangyu Zhao
7. Edoardo Conti
8. Jiliang Tang

In Reinforcement Learning

1. Scott Fujimoto
2. Edoardo Conti
3. Craig Boutilier
4. Long Xia
5. Xing Xie
6. Yi Tay
7. Gerard de Melo
8. Jie Lu

In Recommender Systems

1. Xiang Wang
2. Xiangnan He
3. Yongfeng Zhang
4. Meng Wang
5. Fuli Feng
6. Chuhan Wu
7. Fangzhao Wu
8. Yongfeng Huang

Citations in the corpus

• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform (2018, background): “is high (Chen et al., 2019; Zhao et al., 2018b;a; Mirhoseini et al., 2017; Zheng et al., 2018), the current set of implementations and tooling must be adapted to handle the unique challenges faced in applied settings.”
• Artificial intelligence in recommender systems (2020, background): “Recently, deep reinforcement learning has gained more attention with the breakthrough of deep Q-network and deep deterministic policy gradient, which have advantages in addressing the immediate and long-term rewards simultaneously [148].”
• Deep Learning Based Recommender System (2017, background): “Text Hash Tags [44, 110, 118, 158, 182, 183, 193, 209] News [10, 12, 113, 135, 169, 200] Review texts [11, 87, 126, 146, 174, 197, 202] ‹otes [82, 141] Images Visual features [2, 14, 25, 49, 50, 84, 98, 105, 112, 165, 172, 179, 191, 192, 197, 206] Audio Music [95, 153, 167, 168] Video Videos [14, 17”
• Deep reinforcement learning for page-wise recommendations (2018, background): “Zheng et al.[38] proposed a reinforcement learning framework to do online personalized news recommendation, which can”
• Reinforcement Knowledge Graph Reasoning for Explainable Recommendation (2019, background): “There have been attempts to invoke RL in recommender systems in a non-KG setting, such as for ads recommendation [25], news recommendation [35] and post-hoc explainable recommendation [27].”
• Neural News Recommendation with Long- and Short-term User Representations (2019, background): “Thus, personalized news recommendation is very important for online news platforms to help users find their interested contents and alleviate information overload (Lavie et al., 2010; Zheng et al., 2018).”
• Bias and Debias in Recommender System: A Survey and Future Directions (2020, methodology): “nfidence Bounds (UCB). Some work estimates action-value reward function Q(s;a) with Q network using the Bellman equation and finds the best strategy with the largest function value [148], [149], [150], [151]. Also, the actor network has been adopted recently to learn the best policy by maximizing the ”
• Explainability in Deep Reinforcement Learning (2020, background): “Notable examples include training agents to play Atari games based on raw pixels [75, 76], board games [96, 97], complex real-world robotics problems such as manipulation [8] or grasping [54] and other real-world applications such as resource management in computer clusters [72], network traffic sig”
• FedRec: Privacy-Preserving News Recommendation with Federated Learning (2020, background): “Since massive news articles are posted on-line every day, users of online news services face heavy information overload (Zheng et al., 2018).”
• RecSim: A Configurable Simulation Platform for Recommender Systems (2019, background): “Interest in the application of RL to optimizing these sequences has been rarer [Shani et al., 2005] though the recent successes of deep RL have spurred activity in the use of RL for recommendation [Gauci et al., 2018, Zheng et al., 2018, Chen et al., 2018, Zhao et al., 2018, Ie et al., 2019].”
• Neural News Recommendation with Attentive Multi-View Learning (2019, background): “News recommendation is an important task in natural language processing and data mining fields, and has wide applications [ Zheng et al. , 2018; Wu et al. , 2019a ] .”