Evaluation of session-based recommendation algorithms

By Malte Ludewig, Dietmar Jannach. User modeling and user-adapted interaction, 2018

322 citations recorded by the source corpus.

Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user’s immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like gru4rec, factorized Markov model approaches such as fism or fossil, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today’s more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/7ea0c238d160a6a64611db97429e2b022b552983

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dietmar Jannach
2. Naomie Binti Salim
3. Aminu Da’u
4. Yan Wang
5. Quan Z. Sheng
6. Longbing Cao
7. Shoujin Wang
8. Yi Tay

In Information Systems

1. Dietmar Jannach
2. Naomie Binti Salim
3. Aminu Da’u
4. Longbing Cao
5. Yan Wang
6. Quan Z. Sheng
7. Shoujin Wang
8. Yi Tay

In Management Science and Operations Research

1. Dietmar Jannach
2. Paolo Cremonesi
3. Aminu Da’u
4. Naomie Binti Salim
5. Yan Wang
6. Longbing Cao
7. Quan Z. Sheng
8. Shoujin Wang

Citations in the corpus

• Deep Learning Based Recommender System (2017, background): “RNNs methods can usually lead to best performance. „is work suggests that some baselines in recent works are not well-justi•ed and correctly evaluated. A more comprehensive discussion can be found in [103]. SequentialRecommendationwithUserIdenti•er. Unlikesession-basedrecommenderwhereuseridenti•ers ”
• Recommendation system based on deep learning methods: a systematic review and new directions (2019, background): “Hidasi et al. (2016b), Bansal et al. (2016), Jannach and Ludewig (2017) and Ludewig and Jannach (2018) exploited RNN architecture to better extract features for the user’s immediate next actions recommendation.”
• A Survey on Session-based Recommender Systems (2019, methodology): “Item/session KNN [45, 63] O/UO, ST, A, SL NN Item List-wise deep neural network [127] UO, MT, A, SL MLP Item”
• Sequence-Aware Recommender Systems (2018, background): “A later analysis in [6] for a variety of datasets then revealed that the relative performance of various session-based algorithms strongly depends on the characteristics”
• Continuous Online Evaluation of Recommendation Strategies in Social Science Academic Search (2026, unknown): “Second, more complex models do not necessarily yield better outcomes; simpler methods may provide competitive performance at a fraction of the cost (e.g. semantic similarity vs. CPR)[54].”