A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques

By Jimmy J. Lin, Xueguang Ma. 2021

188 citations recorded by the source corpus.

Recent developments in representational learning for information retrieval can be organized in a conceptual framework that establishes two pairs of contrasts: sparse vs. dense representations and unsupervised vs. learned representations. Sparse learned representations can further be decomposed into expansion and term weighting components. This framework allows us to understand the relationship between recently proposed techniques such as DPR, ANCE, DeepCT, DeepImpact, and COIL, and furthermore, gaps revealed by our analysis point to"low hanging fruit"in terms of techniques that have yet to be explored. We present a novel technique dubbed"uniCOIL", a simple extension of COIL that achieves to our knowledge the current state-of-the-art in sparse retrieval on the popular MS MARCO passage ranking dataset. Our implementation using the Anserini IR toolkit is built on the Lucene search library and thus fully compatible with standard inverted indexes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…on term-weighting schemes, with many recent works proposing sophisticated, contextualized weights for both queries, and documents(Dai and Callan, 2019; Gao et al., 2021; Lin and Ma, 2021; Mallia et al., 2021; Dai and Callan, 2020; Bai et al., 2020; Zhao et al., 2021; Formal et al., 2021b,a).”
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2021, methodology): “We further consider two sparse retriev-445 ers for training Λ, BM25 and UniCOIL (Lin and446 Ma, 2021), a recent state-of-the-art (SoTA) sparse447 retriever, to study whether SPAR training can un-448 pack the knowledge from a more advanced teacher449 model.”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “[170] propose a conceptual framework to unify these approaches: DeepCT aims to assign scalar weights to query terms, while COIL aims to assign vector weights to query terms.”
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction (2022, methodology): “We refer to this model variant as Uni-ColBERTer (following the nomenclature of Lin and Ma [28]).”
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (2024, unknown): “…relevance between query and document is computed based on the interaction score of multiple embeddings (Khattab and Zaharia, 2020), and sparse or lexical retrieval where the importance of each term is estimated by its output embedding (Gao et al., 2021a; Lin and Ma, 2021; Dai and Callan, 2020).”
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “Most modern information retrieval systems are based on the index-then-retrieve pipeline where documents are first encoded and stored in an inverted index [10, 13, 15, 26, 34, 55, 58] or a nearest neighbor search index [7, 14, 20, 21, 25, 29, 49, 51] and search results are then constructed based on a”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “The uniCOIL (Lin and Ma, 2021) model simplifies COIL by limiting its token embedding vector to a single dimension, which in effect replaces the vector weights with scalar weights.”
A Systematic Multi-Domain Evaluation of Document Retrievers (2026, unknown): “Retrievers. docT5query [58] expands documents by generating multiple queries for each and retrieving with BM25. query2doc [80] instead expands the query, generating a pseudo-document and again retrieving with BM25, while HyDE [19] generates a hypothetical document and retrieves with the Contriever.”
Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Learned sparse retrieval incorporates neural networks to generate sparse, term-based representations [39, 58, 139, 161].”
A Survey of Model Architectures in Information Retrieval (2025, unknown): “(Gao et al., 2021b,a; Ma et al., 2021; Lin and Ma, 2021; Cormack et al., 2009).”