Recommender Systems with Generative Retrieval

By Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan H. Keshavan, T. Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost and 3 more. Neural Information Processing Systems, 2023

342 citations recorded by the source corpus.

Modern recommender systems perform large-scale retrieval by first embedding queries and item candidates in the same unified space, followed by approximate nearest neighbor search to select top candidates given a query embedding. In this paper, we propose a novel generative retrieval approach, where the retrieval model autoregressively decodes the identifiers of the target candidates. To that end, we create semantically meaningful tuple of codewords to serve as a Semantic ID for each item. Given Semantic IDs for items in a user session, a Transformer-based sequence-to-sequence model is trained to predict the Semantic ID of the next item that the user will interact with. To the best of our knowledge, this is the first Semantic ID-based generative model for recommendation tasks. We show that recommender systems trained with the proposed paradigm significantly outperform the current SOTA models on various datasets. In addition, we show that incorporating Semantic IDs into the sequence-to-sequence model enhances its ability to generalize, as evidenced by the improved retrieval performance observed for items with no prior interaction history.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems

View Paper: https://www.semanticscholar.org/paper/a289100678e7d94af836d91cd48d7821ebc5b83d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wayne Xin Zhao
2. Ji-rong Wen
3. Yupeng Hou
4. Hongyu Lu
5. Bowen Zheng
6. Yu Chen
7. Wenqi Fan
8. Yiqi Wang

In Computer Vision and Pattern Recognition

1. Ji-rong Wen
2. Wayne Xin Zhao
3. Yupeng Hou
4. Jiliang Tang
5. Yiqi Wang
6. Hongyu Lu
7. Bowen Zheng
8. Yu Chen

In Information Systems

1. Wayne Xin Zhao
2. Yupeng Hou
3. Ji-rong Wen
4. Hongyu Lu
5. Bowen Zheng
6. Yu Chen
7. Jiliang Tang
8. Xiangnan He

Citations in the corpus

• Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2023, methodology): “To address this issue, we borrow the idea of existing studies [15], [23], [37] to learn indices associated with latent semantics for items.”
• Retrieval-Augmented Generation for Large Language Models: A Survey (2023, background): “Discerning the importance and relevance of multiple retrieved passages to the generation task is another challenge, requiring the proper balance of each passage’s value.”
• Recommender Systems in the Era of Large Language Models (LLMs) (2023, positive): “Meanwhile, to generate these codewords, a hierarchical method called RQ-VAE is also proposed [64] to leverage Semantic IDs, where recommendation data formats can be effectively transformed into natural language sequences for transformer-based models.”
• A survey on large language models for recommendation (2023, neutral): “Addressing these fairness issues is crucial to ensure equitable and unbiased recommendations.”
• Open-Source Embedding Models: A Comprehensive Survey of Techniques, Benchmarks, and Applications (2026, unknown): “The most profound re-engineering of the retrieval process is the emergence of generative retrieval [106], [107].”