Collaborative Filtering and the Missing at Random Assumption

By Benjamin M. Marlin, Richard S. Zemel, Sam T. Roweis, Malcolm Slaney. 2007

336 citations recorded by the source corpus.

Rating prediction is an important application, and a popular research topic in collaborative filtering. However, both the validity of learning algorithms, and the validity of standard testing procedures rest on the assumption that missing ratings are missing at random (MAR). In this paper we present the results of a user study in which we collect a random sample of ratings from current users of an online radio service. An analysis of the rating data collected in the study shows that the sample of random ratings has markedly different properties than ratings of user-selected songs. When asked to report on their own rating behaviour, a large number of users indicate they believe their opinion of a song does affect whether they choose to rate that song, a violation of the MAR condition. Finally, we present experimental results showing that incorporating an explicit model of the missing data mechanism can lead to significant improvements in prediction performance on the random sample of ratings.

Subfields: Computer Science Applications, Information Systems, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Signal Processing

Citations in the corpus

Recommendations as Treatments: Debiasing Learning and Evaluation (2016, methodology): “Past work that explicitly dealt with the MNAR nature of recommendation data approached the problem as missingdata imputation based on the joint likelihood of the missing data model and the rating model (Marlin et al., 2007; Marlin & Zemel, 2009; Hernández-Lobato et al., 2014).”
Offline A/B Testing for Recommender Systems (2018, background): “click) can be translated into a supervised task [10, 16, 22].”
Training and testing of recommender systems on data missing not at random (2010, background): “This kind of data is inexpensive to collect, and typically used for training and testing recommender systems (prior to conducting more costly (cid:12)eld studies).”
Mind the gaps: weighting the unknown in large-scale one-class collaborative filtering (2009, background): “In [2] and [12], the authors discuss the issue of modeling the distribution of missing values.”
Graphical Models for Inference with Missing Data (2013, background): “Exceptions are recent work on collaborative filtering and recommender systems which develops probabilistic models that explicitly incorporate missing data mechanism [16, 14, 15].”
Probabilistic Matrix Factorization with Non-random Missing Data (2014, background): “However, there is evidence that CF data is missing not at random (MNAR) (Marlin & Zemel, 2007).”
One-Class Collaborative Filtering (2008, background): “In [2] and [19], the authors discuss the issue of modeling the distribution of missing values in collaborative filtering problems.”
Personalized recommendation via cross-domain triadic factorization (2013, background): “1 Confidence Level In fact, implicit feedbacks can indirectly reflect opinions through user behavior because users may deliberately choose to access which items [17].”
Disentangling User Interest and Conformity for Recommendation with Causal Embedding (2021, background): “Existing causal solutions for recommender systems formulate the problem as eliminating popularity bias, from the perspective of items [1, 3, 5, 11, 12, 24, 35, 43].”
Scalable Recommendation with Hierarchical Poisson Factorization (2015, background): “In addition to the difﬁculty in appropriately weighting or sampling negative examples, there is a known selection bias in provided ratings that causes further complications [28].”
How should I explain? A comparison of different explanation types for recommender systems (2014, background): “McSherry [2005], in contrast, measures efficiency through the number of required dialogue steps before a user accepts one of the system’s recommendations.”
New Fairness Metrics for Recommendation that Embrace Differences (2017, background): “Unfortunately, research has shown that sampled ratings have markedly different properties from the users’ true preferences [16, 17], which is a potential source of unfairness.”
Recommender systems and their ethical challenges (2020, background): “However, this assumption of randomness introduces a potential source of bias in the system’s predictions, because it is well documented that users’ underlying preferences often differ from the sampled ratings, since the latter are affected by social factors, which may be biased (Marlin et al. 2007).”
Towards Deep Conversational Recommendations (2018, background): “This is standard for recommendation data [20], since people are naturally more likely to talk about movies that they like, and the recommender’s objective is to recommend movies that the seeker is likely to like.”
Beyond Parity: Fairness Objectives for Collaborative Filtering (2017, background): “[2, 21, 25] that use side information to address the problem of imbalanced data, but in this work, to test the properties and effectiveness of our metrics, we focus on the basic matrix-factorization algorithm first.”
Evaluating Recommendation Systems (2011, background): “Thus, the set of items on which explicit ratings are available may be biased by the ratings themselves [38].”
Fast Matrix Factorization for Online Recommendation with Implicit Feedback (2016, background): “[18] finds that users listen to music they expect to like and avoid the genres they dislike, leading to a severe bias in the observed data.”
Modeling User Exposure in Recommendation (2015, neutral): “[16], Ling et al.”