SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking

By Thibault Formal, Benjamin Piwowarski, Stéphane Clinchant. 2021

611 citations recorded by the source corpus.

In neural Information Retrieval, ongoing research is directed towards improving the first retriever in ranking pipelines. Learning dense embeddings to conduct retrieval using efficient approximate nearest neighbors methods has proven to work well. Meanwhile, there has been a growing interest in learning sparse representations for documents and queries, that could inherit from the desirable properties of bag-of-words models such as the exact matching of terms and the efficiency of inverted indexes. In this work, we present a new first-stage ranker based on explicit sparsity regularization and a log-saturation effect on term weights, leading to highly sparse representations and competitive results with respect to state-of-the-art dense and sparse methods. Our approach is simple, trained end-to-end in a single stage. We also explore the trade-off between effectiveness and efficiency, by controlling the contribution of the sparsity regularization.

Subfields: Artificial Intelligence, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…on term-weighting schemes, with many recent works proposing sophisticated, contextualized weights for both queries, and documents(Dai and Callan, 2019; Gao et al., 2021; Lin and Ma, 2021; Mallia et al., 2021; Dai and Callan, 2020; Bai et al., 2020; Zhao et al., 2021; Formal et al., 2021b,a).”
SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval (2021, methodology): “In this section, we first describe in details the SPLADE model recently introduced in [8].”
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “Most modern information retrieval systems are based on the index-then-retrieve pipeline where documents are first encoded and stored in an inverted index [10, 13, 15, 26, 34, 55, 58] or a nearest neighbor search index [7, 14, 20, 21, 25, 29, 49, 51] and search results are then constructed based on a”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “…distillation from cross-encoders, namely, ColBERT (Khattab and Zaharia, 2020), DPR-MARCO (Xin et al., 2021), ANCE (Xiong et al., 2020), and MoDIR (Xin et al., 2021), as well as models that do utilize distillation, namely, TAS-B (Hofstätter et al., 2021) and SPLADEv2 (Formal et al., 2021a).”
Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers (2026, unknown): “MoE retrievers consistently lie near the Pareto frontier (shaded). as BM25 rank documents by term overlap, while learned sparse methods such as SPLADE expand and reweight terms to reduce vocabulary mismatch (Robertson and Zaragoza, 2009; Formal et al., 2021).”
What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models (2025, unknown): “Sparse neural retrievers such as SPLADE [26] and SPLADE-v2 [27] enhance lexical matching via learned expansion, while dense retrieval approaches embed queries and documents into a shared vector space using multilingual encoders and rank via cosine similarity [5, 28, 29].”
From Semantic Retrieval to Conversational Agent: A Web-Based RAG Architecture for Interactive System Dynamics Modeling (2026, unknown): “When combined with modern hybrid and sparse-expansion retrieval techniques [23], these methodologies address the inherent limitations of passive information seeking.”
Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework (2026, unknown): “…retrieval represents each piece of text as a very high-dimensional vector — one dimension per vocabulary term — where almost all values are zero and non-zero values indicate term importance (classical term-weighting schemes like BM25 [27], or learned variants such as SPLADE [26], work this way).”
HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions (2026, unknown): “…Robertson and Zaragoza, 2009; Thakur et al., 2021), dense retrieval with bi-encoders (Karpukhin et al., 2020; Reimers and Gurevych, 2019), learned sparse SPLADE-family models (Formal et al., 2021), and token-level late interaction (ColBERT family; Khattab and Zaharia, 2020; Santhanam et al., 2021).”
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies (2026, unknown): “Learned sparse retrievers such as SPLADE expose sparse vocabulary weights that can be indexed and searched efficiently (Formal et al., 2021b).”
Granite Embedding Models (2025, unknown): “…could be useful for exact matching of terms and could be efficiently ingested as an inverted index as shown by Zamani et Sparse models are popularly trained to predict the importance of each vocabulary term based on the logits of masked-language model head (Bai et al., 2020; Formal et al., 2021b).”
On the Challenges and Opportunities of Learned Sparse Retrieval for Code (2026, unknown): “Most high-performing LSR models are based on SPLADE [7].”
A Multimodal and Multilingual Offline Retrieval-Augmented Generation System Using Local Large Language Models (2026, unknown): “[16] for late interaction, and SPLADE [17] for sparse neural representations—all requiring GPU inference.”