Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning

By Xiangyu Zhao, L. Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, Dawei Yin. Knowledge Discovery and Data Mining, 2018

386 citations recorded by the source corpus.

Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedback. Users' feedback can be positive and negative and both types of feedback have great potentials to boost recommendations. However, the number of negative feedback is much larger than that of positive one; thus incorporating them simultaneously is challenging since positive feedback could be buried by negative one. In this paper, we develop a novel approach to incorporate them into the proposed deep recommender system (DEERS) framework. The experimental results based on real-world e-commerce data demonstrate the effectiveness of the proposed framework. Further experiments have been conducted to understand the importance of both positive and negative feedback in recommendations.

Subfields: Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/ef08ad10d257fcf5043d27d7225d6ec14a677a84

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Bias and Debias in Recommender System: A Survey and Future Directions (2020, methodology): “Some work estimates action-value reward function Q(s, a) with Q network using the Bellman equation and finds the best strategy with the largest function value [148], [149], [150], [151].”
Reinforcement Learning based Recommender Systems: A Survey (2021, methodology): “DRL-based Methods RLRS Year SR PO RF EB Metrics Dataset Application Code Slate-MDP [112] 2015 SR1 DQN R1 Simulation Reward N/A Wolpertinger [24] 2015 SR1 DDPG R1 Simulation Reward N/A Nemati et al. [113] 2016 SR2 DQN R2 Oline Reward MIMIC Healthcare CEI [114] 2017 SR3 REINFORCE-wb R1 Simulation P, F”
Deep Reinforcement Learning for List-wise Recommendations (2017, background): “Recommender systems have become increasingly popular in recent years, and have been utilized in a variety of domains including movies, music, books, point of interests, and social events[5, 21, 22, 32–36].”
Deep Learning Based Recommender System (2017, methodology): “[200] proposed a DRL framework, DEERS, for recommendation with both negative and positive feedback in a sequential interaction setting.”
Graph Neural Networks for Social Recommendation (2019, background): “Some recent efforts have applied deep neural networks to recommendation tasks and shown promising results [41], but most of them used deep neural networks to model audio features of music [32], textual description of items [3, 33], and visual content of images [40].”
Deep reinforcement learning for page-wise recommendations (2018, methodology): “Following [42], we build a simulated online environment (adapted to our case) for online test.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, methodology): “Model-free frameworks such as deep Q-network (DQN) [213, 215, 230, 225] and deep deterministic policy gradient (DDPG) [65] are used in interactive recommendation scenarios.”