Evaluating Recommendation Systems

By Guy Shani, Asela Gunawardana. 2011

1,451 citations recorded by the source corpus.

Subfields: Computer Science Applications, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Explainable Recommendation: A Survey and New Perspectives (2018, background): “, 2011), and evaluation of recommender systems (Shani and Gunawardana, 2011).”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “When biases cannot be ruled out as the biasing factors are too complex or hard to untangle, running longitudinal, multi-datasets, cross-domain or platform analyses may be needed (Shani and Gunawardana, 2011; Schoen et al., 2013; Ruths and Pfeﬀer, 2014).”
TFMAP: optimizing MAP for top-n context-aware recommendation (2012, background): “It is thus a good measure of performance when a short list of the most relevant items is shown to users [27].”
Recommender Systems (2016, neutral): “Excellent discussions on evaluating recommender systems may be found in [246, 275, 538].”
A probabilistic model to resolve diversity–accuracy challenge of recommendation systems (2015, methodology): “It has been shown that performance of the recommender systems should be evaluated in terms of not only accuracy but also diversity and novelty [22, 36, 37].”
A survey of active learning in collaborative filtering recommender systems (2016, background): “It is implicitly assumed that the simulated user behaviour in offline experiment is similar to that actually shown by them while interacting with a real deployed system [92].”
Controlling Popularity Bias in Learning-to-Rank Recommendation (2017, background): “Recommending serendipitous items from the long tail are generally considered to be valuable to users [2, 16], as these are items that users are less likely to know about.”
Domain-specific Knowledge Extraction from the Web of Data (2018, methodology): “precision@n in comparison to other metrics such as recall is the most suitable evaluation metric for recommender systems, particularly where the number of recommended items are pre-obtained [Shani and Gunawardana 2011].”
A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method (2016, methodology): “Hence, we choose the root mean squared error (RMSE) and mean absolute error (MAE) [44], [45] as the accuracy metrics, both of which are widely used for evaluating the statistical accuracy of CF models.”
The MovieLens Datasets: History and Context (2016, methodology): “…ratings scales, rescaled for the .nal dataset. cIncludes ratings for songs, albums, artists, and genres. d98% of rating values are multiples of 10. the online system that they wish to simulate [Shani and Gunawardana 2011], or if they wish to evaluate an approach across multiple datasets.”
Collaborative Filtering Recommender Systems (2011, background): “Shani and Gunawardana [135] describe many aspects of online and offline evaluations, and more work is needed to come up with good ways to assess recommender’s ability to meet user’s needs.”
NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning (2021, background): “, in recommender systems for supervised learning models [9].”
Calibrated recommendations (2018, methodology): “In the literature of recommender systems, the focus has been on various metrics besides accuracy, e.g., see [21] for an overview, among which diversity is closest to calibration.”
Precision-oriented evaluation of recommender systems: an algorithmic comparison (2011, methodology): “We use Precision, Recall, and normalized Discounted Cumulative Gain (nDCG) [10] as representative precisionoriented metrics.”