Reinforcement Learning based Recommender Systems: A Survey

By M. M. Afsar, Trafford Crump, Behrouz Homayoun Far. ACM Computing Surveys, 2021

605 citations recorded by the source corpus.

Recommender systems (RSs) have become an inseparable part of our everyday lives. They help us find our favorite items to purchase, our friends on social networks, and our favorite movies to watch. Traditionally, the recommendation problem was considered to be a classification or prediction problem, but it is now widely agreed that formulating it as a sequential decision problem can better reflect the user-system interaction. Therefore, it can be formulated as a Markov decision process (MDP) and be solved by reinforcement learning (RL) algorithms. Unlike traditional recommendation methods, including collaborative filtering and content-based filtering, RL is able to handle the sequential, dynamic user-system interaction and to take into account the long-term user engagement. Although the idea of using RL for recommendation is not new and has been around for about two decades, it was not very practical, mainly because of scalability problems of traditional RL algorithms. However, a new trend has emerged in the field since the introduction of deep reinforcement learning (DRL), which made it possible to apply RL to the recommendation problem with large state and action spaces. In this paper, a survey on reinforcement learning based recommender systems (RLRSs) is presented. Our aim is to present an outlook on the field and to provide the reader with a fairly complete knowledge of key concepts of the field. We first recognize and illustrate that RLRSs can be generally classified into RL- and DRL-based methods. Then, we propose an RLRS framework with four components, i.e., state representation, policy optimization, reward formulation, and environment building, and survey RLRS algorithms accordingly. We highlight emerging topics and depict important trends using various graphs and tables. Finally, we discuss important aspects and challenges that can be addressed in the future.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/5685abf9e7bb2c16449ae1eb181051e503602a55

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Advances and Challenges in Conversational Recommender Systems: A Survey (2021, positive): “Feasible directions include designing systematic simulation agenda [246, 172], building dense user interactions for reliable simulation [275, 29, 5], andmodeling user choice behaviors over the slate recommendation [75, 135, 2].”
Harms from Increasingly Agentic Algorithmic Systems (2023, background): “Additionally, there has been an increasing amount of research in recommender systems to optimize long-term metrics such as engagement via reinforcement learning [Afsar et al. 2022].”
Recommender Systems in the Era of Large Language Models (LLMs) (2023, neutral): “In this section, we briefly review some related work on recommender systems and LLMs techniques.”