ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

By Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, Matei A. Zaharia. 2021

784 citations recorded by the source corpus.

Neural information retrieval (IR) has greatly advanced search and other knowledge-intensive language tasks. While many neural IR methods encode queries and documents into single-vector representations, late interaction models produce multi-vector representations at the granularity of each token and decompose relevance modeling into scalable token-level computations. This decomposition has been shown to make late interaction more effective, but it inflates the space footprint of these models by an order of magnitude. In this work, we introduce Maize, a retriever that couples an aggressive residual compression mechanism with a denoised supervision strategy to simultaneously improve the quality and space footprint of late interaction. We evaluate Maize across a wide range of benchmarks, establishing state-of-the-art quality within and outside the training domain while reducing the space footprint of late interaction models by 6–10x.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Efficiency & Compression

Citations in the corpus

Landmark Attention: Random-Access Infinite Context Length for Transformers (2023, methodology): “Previous work has also looked into using the attention in the reader to build a retriever but used manually crafted rules to reduce the token scores to document scores [14, 18, 31].”
Memory-Based Model Editing at Scale (2022, methodology): “While other more sophisticated approaches exist (Khattab & Zaharia, 2020; Santhanam et al., 2021), we restrict our experiments to either cross-attention ( Cross ) or embedding-based ( Embed ) scope classiﬁers.”
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer (2022, methodology): “…illustrate the required computation, where close documents require both retrieval and target attention while random documents only require retrieval attention. in index compression and search approximation to make test-time retrieval efficient, which we refer to Santhanam et al. (2021) for details.”
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP (2022, methodology): “5 and our retrieval-based rows use ColBERTv2.”
Promptagator: Few-shot Dense Retrieval From 8 Examples (2022, methodology): “P ROMPTA - GATOR outperforms ColBERT v2 and SPLADE v2 on 11 retrieval tasks we tested, while reranking boosts results by another 5 points on standard retrieval evaluation metric.”
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction (2022, background): “[43] study residual compression of all saved token vectors as part of the ColBERT end-to-end retrieval setting.”
Dialog Inpainting: Turning Documents into Dialogs (2022, background): “Previous work has identified several weakness of in-batch negatives and identified the importance of selecting challenging negative examples for training retrievers (Qu et al., 2021; Xiong et al., 2021; Santhanam et al., 2021; Lin et al., 2021).”
Task-aware Retrieval with Instructions (2022, methodology): “To evaluate the models’ ability to perform zeroshot transfer via instructions, we run experiments on two widely used zero-shot transfer evaluation benchmarks: BEIR (Thakur et al., 2021) and LOTTE-pooled (Santhanam et al., 2022).”
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval (2022, methodology): “1,Mean-Max strategies [25, 26, 47, 59] ignore the importance of different frames and words.”
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (2024, neutral): “We follow the practice of previous studies (De-vlin et al., 2019; Liu et al., 2019a; He et al., 2023) and conduct a hyperparameter search on each GLUE subset (detailed in Appendix E.1) in order to provide values comparable to other models.”
Extending Context Window of Large Language Models via Positional Interpolation (2023, background): “One line of work extends LLMs by augmenting it with retrieval modules which fetch related documents and include the retrieval results into the input context of an LLM (Karpukhin et al., 2020; Guu et al., 2020; Izacard et al., 2022; Jiang et al., 2022; Khattab et al., 2021; Santhanam et al., 2022).”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, methodology): “Implementation To improve generality, we avoid relying on task-specific retrievers (Santhanam et al., 2022; Khattab et al., 2022) that may lead to higher performance and overfitting.”
IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions (2026, unknown): “More recently, methods such as late interaction (Santhanam et al., 2022; Jha et al., 2024) and late Chunking (Günther et al., 2025) aim to preserve contextual information while enabling localized matching.”
TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks (2025, unknown): “Turkish-ColBERT [21] adapted the ColBERT architecture, initially designed for English information retrieval, to the Turkish language, showcasing the adapt-ability of advanced retrieval models to morphologically complex languages.”
NEAR: Neighborhood-Aware Residuals for Memory Retrieval (2026, unknown): “ColBERTv2 and XTR retain their native encoders and input settings.”