Making sense of recommendations

By Michael Yeomans, Anuj K. Shah, Sendhil Mullainathan, Jon M. Kleinberg. 2019

401 citations recorded by the source corpus.

. Algorithms are increasingly being used to make recommendations about matters of taste, expanding their scope into domains that are primarily subjective. This raises two important questions. How accurately can algorithms predict subjective preferences, compared to human recommenders? And how much do people trust them? Recommender systems face several disadvantages: They have no preferences of their own and they do not model their recommendations after the way people make recommendations. In a series of experiments, however, we find that recommender systems outperform human recommenders, even in a domain where people have a lot of experience and well-developed tastes: Predicting what people will find funny. Moreover, these recommender systems outperform friends, family members, and significant others. But people do not trust these recommender systems. They do not use them to make recommendations for others, and they prefer to receive recommendations from other people instead. We find that this lack of trust partly stems from the fact that machine recommendations seem harder to understand than human recommendations. But, simple explanations of recommender systems can alleviate this distrust.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Recommender Systems

Citations in the corpus

How To Backdoor Federated Learning (2018, background): “Many users trust machine-provided recommendations [70] and their online behavior can be influenced by what they see [35].”
Task-Dependent Algorithm Aversion (2019, background): “Finally, Yeomans et al. (2019) found that participants relied less on an algorithm than on humans for the task of predicting joke funniness, while Logg, Minson, and Moore (2019) found that participants relied more on algorithms than on humans for numerical tasks with an objectively correct answer.”
The Principles and Limits of Algorithm-in-the-Loop Decision Making (2019, background): “People often discount algorithmic recommendations, preferring to rely on their own or other people’s judgment and exhibiting less tolerance for errors made by algorithms than errors made by other people [22, 56, 76].”
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments (2019, background): “Even when algorithms are more accurate, people do not appropriately incorporate algorithmic recommendations to improve their decisions, instead preferring to rely on their own or other people’s judgment [47, 66].”
Resistance to Medical Artificial Intelligence (2019, background): “…to rely on friends rather than on computerized recommendation systems for suggestions about books, movies, and jokes (Sinha and Swearingen 2001; Yeomans, Shah, Mullainathan, and Kleinberg, 2019) , lose confidence in statistical models more quickly than in humans after seeing both make the same…”
Understanding the Effect of Accuracy on Trust in Machine Learning Models (2019, background): “[30] found that people distrust automated recommender systems compared with human recommendations in the context of predicting which jokes people will find funny—a highly subjective domain—even when the recommender system outperforms human predictions.”
Economics, Fairness and Algorithmic Bias (2020, neutral): “A nascent literature in psychology and economics (Yeomans et al., 2017; Bigman and Gray, 2018) seeks to explain when and why humans defer to algorithms, with some researchers reporting boundary conditions to “algorithm aversion,” particularly for non-experts (Logg et al., 2018).”
Algorithm appreciation: People prefer algorithmic to human judgment (2019, neutral): “It may be perfectly reasonable to rely on the advice of close friends rather than a “black box” algorithm when making a decision reflecting one’s own personal taste (Yeomans et al., 2018).”