SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval

By Thibault Formal, C. Lassance, Benjamin Piwowarski, Stéphane Clinchant. 2021

247 citations recorded by the source corpus.

In neural Information Retrieval (IR), ongoing research is directed towards improving the first retriever in ranking pipelines. Learning dense embeddings to conduct retrieval using efficient approximate nearest neighbors methods has proven to work well. Meanwhile, there has been a growing interest in learning \emph{sparse} representations for documents and queries, that could inherit from the desirable properties of bag-of-words models such as the exact matching of terms and the efficiency of inverted indexes. Introduced recently, the SPLADE model provides highly sparse representations and competitive results with respect to state-of-the-art dense and sparse approaches. In this paper, we build on SPLADE and propose several significant improvements in terms of effectiveness and/or efficiency. More specifically, we modify the pooling mechanism, benchmark a model solely based on document expansion, and introduce models trained with distillation. We also report results on the BEIR benchmark. Overall, SPLADE is considerably improved with more than $9$\% gains on NDCG@10 on TREC DL 2019, leading to state-of-the-art results on the BEIR benchmark.

Subfields: Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…on term-weighting schemes, with many recent works proposing sophisticated, contextualized weights for both queries, and documents(Dai and Callan, 2019; Gao et al., 2021; Lin and Ma, 2021; Mallia et al., 2021; Dai and Callan, 2020; Bai et al., 2020; Zhao et al., 2021; Formal et al., 2021b,a).”
Unsupervised Dense Information Retrieval with Contrastive Learning (2021, methodology): “For sparse methods, we compare to Splade v2 (Formal et al., 2021), which computes sparse representations of documents with BERT pre-trained model.”
SGPT: GPT Sentence Embeddings for Semantic Search (2022, methodology): “However, they have been limited to BERT-like encoder-only transformers [26, 54, 10, 39, 12, 29].”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, background): “In addition, SPLADE [172] and SPLADEv2 [173] project each term in queries and texts to a vocabularysized weight vector, in which each dimension represents the weight of a term in the vocabulary of PLMs.”
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction (2022, background): “There are concurrent efforts revisiting lexical matching with learned sparse representations [10, 12, 26] or learned passage impacts [37], which employ the efficiency of exact lexical matches.”
Promptagator: Few-shot Dense Retrieval From 8 Examples (2022, methodology): “P ROMPTA - GATOR outperforms ColBERT v2 and SPLADE v2 on 11 retrieval tasks we tested, while reranking boosts results by another 5 points on standard retrieval evaluation metric.”
Text and Code Embeddings by Contrastive Pre-Training (2022, methodology): “In the transfer setting, our models achieve a 5.2% relative improvement over previous methods (Izacard et al., 2021) and is comparable even with methods (Santhanam et al., 2021; Formal et al., 2021; Wang et al., 2020) that demand substantially more computation at test time.”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “While ColBERT outperforms single-vector systems like RepBERT, ANCE, and even TAS-B, improvements in supervision such as distillation from cross-encoders (see §2) enable systems like SPLADEv2, PAIR, and RocketQAv2 to achieve higher quality than vanilla ColBERT.”
Large Dual Encoders Are Generalizable Retrievers (2021, background): “There have been shown distillation is a strong recipe to improve the dense retrieval models on out-of-domain performance (Santhanam et al., 2021; Formal et al., 2021).”
What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models (2025, unknown): “Sparse neural retrievers such as SPLADE [26] and SPLADE-v2 [27] enhance lexical matching via learned expansion, while dense retrieval approaches embed queries and documents into a shared vector space using multilingual encoders and rank via cosine similarity [5, 28, 29].”
Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems (2026, unknown): “We do not benchmark learned sparse retrievers such as SPLADE-v2 (Formal et al., 2021) because multilingual checkpoints for this family remain scarce.”
Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Learned sparse retrieval incorporates neural networks to generate sparse, term-based representations [39, 58, 139, 161].”
Granite Embedding Models (2025, unknown): “We follow Formal et al. (2021a) and use max-pooling to compute the importance w j for each vocabulary token j for input sequence I : Learning Sparse Representations We observe that minimizing the FLOPS loss Paria et al. (2020), as shown in SPLADE (Formal et al., 2021b;a), did not lead to a…”
A Survey of Model Architectures in Information Retrieval (2025, unknown): “Drawing from the evolution of IR architectures (§§ 3 to 7), we examine emerging trends, open problems, and potential research directions.”