Short-Term Satisfaction and Long-Term Coverage: Understanding How Users Tolerate Algorithmic Exploration

By Tobias Schnabel, Paul N. Bennett, Susan T. Dumais, Thorsten Joachims. 2018

53 citations recorded by the source corpus.

Any learning algorithm for recommendation faces a fundamental trade-off between exploiting partial knowledge of a user»s interests to maximize satisfaction in the short term and discovering additional user interests to maximize satisfaction in the long term. To enable discovery, a machine learning algorithm typically elicits feedback on items it is uncertain about, which is termed algorithmic exploration in machine learning. This exploration comes with a cost to the user, since the items an algorithm chooses for exploration frequently turn out to not match the user»s interests. In this paper, we study how users tolerate such exploration and how presentation strategies can mitigate the exploration cost. To this end, we conduct a behavioral study with over 600 people, where we vary how algorithmic exploration is mixed into the set of recommendations. We find that users respond non-linearly to the amount of exploration, where some exploration mixed into the set of recommendations has little effect on short-term satisfaction and behavior. For long-term satisfaction, the overall goal is to learn via exploration about the items presented. We therefore also analyze the quantity and quality of implicit feedback signals such as clicks and hovers, and how they vary with different amounts of mix-in exploration. Our findings provide insights into how to design presentation strategies for algorithmic exploration in interactive recommender systems, mitigating the short-term costs of algorithmic exploration while aiming to elicit informative feedback data for learning.

Subfields: Computer Science Applications, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Making deep neural networks right for the right scientific reasons by interacting with their explanations (2020, background): “There is a trade-off between query informativeness and user “satisfaction”, as noticed and explored in [46].”
Top-K Off-Policy Correction for a REINFORCE Recommender System (2018, unknown): “[35] studied the cost of exploration.”
Optimal Decision Making Under Strategic Behavior (2019, result): “ntributions are orthogonal to ours. More broadly, our work also relates to several recent studies on the long-term consequences of machine learning algorithms [10, 15, 17, 21] and recommender systems [19, 20]. Once we focus on the utility of a decision policy, it is overly pessimistic to always view”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “yet, recommendations with low confidence should not be exposed to the user (Schnabel et al., 2018).”
A Survey on Session-based Recommender Systems (2019, background): “In practice, such implicit preference data is often available in the session-based recommdation scenario [91].”