Dense Text Retrieval Based on Pretrained Language Models: A Survey
By Wayne Xin Zhao, Jing Liu, Ruiyang Ren, Ji-rong Wen. ACM Trans. Inf. Syst., 2022
325 citations recorded by the source corpus.
Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user’s queries in natural language. From heuristic-based retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn text representations and model the relevance matching. The recent success of pretrained language models (PLM) sheds light on developing more capable text-retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the semantic representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is called dense retrieval, since it employs dense vectors to represent the texts. Considering the rapid progress on dense retrieval, this survey systematically reviews the recent progress on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related studies by four major aspects, including architecture, training, indexing and integration, and thoroughly summarize the mainstream techniques for each aspect. We extensively collect the recent advances on this topic, and include 300+ reference papers. To support our survey, we create a website for providing useful resources, and release a code repository for dense retrieval. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/3a07a87090a061ca41dd30ac8398a9a5d9d39826
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
Citations in the corpus
- mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024, background): “It is often implemented as a multi-stage process, consisting of two main components: a re-triever and a reranker (Gao et al., 2021a; Zhang et al., 2022; Zhao et al., 2024).”
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, unknown): “To mitigate these limitations, search engines (Zhao et al., 2024) are widely integrated to supply external information.”
- Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “The evidence retrieval process typically employs various sparse (e.g., BM25 (Robertson et al., 2009)) or dense (e.g., PLM-based meth-ods (Zhao et al., 2022)) retrievers.”
- PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval (2021, background): “Different from sparse retrieval, dense passage retrieval represents queries and passages into lowdimensional vectors (Zhao et al., 2022; Guu et al., 2020; Karpukhin et al., 2020), typically in a dual-”
- Large Language Models for Information Retrieval: A Survey (2023, unknown): “In recent years, the neural IR [30, 55, 56] paradigm has gained considerable”
- Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “In contrast to sparse lexical methods, dense retrieval [61, 304, 314] uses continuous vector embeddings to represent queries and documents.”
- Length-Induced Embedding Collapse in PLM-based Models (2024, unknown): “…of texts—are essential for many NLP applications, including text analysis (Aggarwal and Zhai, 2012; Angelov, 2020), question answering (Tan et al., 2023; Xu et al., 2024), web search (Zhao et al., 2024; Yates et al., 2021), and retrieval-augmented generation (Gao et al., 2023; Fan et al., 2024).”