COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List

By Luyu Gao, Zhuyun Dai, Jamie Callan. 2021

275 citations recorded by the source corpus.

Classical information retrieval systems such as BM25 rely on exact lexical match and can carry out search efficiently with inverted list index. Recent neural IR models shifts towards soft matching all query document terms, but they lose the computation efficiency of exact match systems. This paper presents COIL, a contextualized exact match retrieval architecture, where scoring is based on overlapping query document tokens’ contextualized representations. The new architecture stores contextualized token representations in inverted lists, bringing together the efficiency of exact match and the representation power of deep language models. Our experimental results show COIL outperforms classical lexical retrievers and state-of-the-art deep LM retrievers with similar or smaller latency.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…on term-weighting schemes, with many recent works proposing sophisticated, contextualized weights for both queries, and documents(Dai and Callan, 2019; Gao et al., 2021; Lin and Ma, 2021; Mallia et al., 2021; Dai and Callan, 2020; Bai et al., 2020; Zhao et al., 2021; Formal et al., 2021b,a).”
SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval (2021, background): “Furthermore, COIL [9] proposed to revisit exact-match mechanisms by learning dense representations per term to perform contextualized term matching, at the cost of increased index size.”
Promptagator: Few-shot Dense Retrieval From 8 Examples (2022, methodology): “The other approach provides the LLM the instruction, a few examples and also performs model ﬁne-tuning (Schick & Schütze, 2021a;b;c; Gao et al., 2021b; Logan IV et al., 2022; Izacard et al., 2022b).”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “The uniCOIL (Lin and Ma, 2021) model simplifies COIL by limiting its token embedding vector to a single dimension, which in effect replaces the vector weights with scalar weights.”
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval (2021, background): “Essentially, the three systems represent three distinct and equally good approaches for effective web passage retrieval: optimized dense retrieval ﬁne-tuning in RocketQA (Qu et al., 2021), contextualized sparse retrieval in COIL (Gao et al., 2021a), and corpus-aware un-supervised pre-training in coC”
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction (2022, methodology): “Wellstudied low hanging fruits for good tradeoffs include reducing the number of dimensions (either inside the model or outside with PCA) and reducing the number of bytes with quantization [12, 21, 27, 36].”
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (2024, unknown): “…relevance between query and document is computed based on the interaction score of multiple embeddings (Khattab and Zaharia, 2020), and sparse or lexical retrieval where the importance of each term is estimated by its output embedding (Gao et al., 2021a; Lin and Ma, 2021; Dai and Callan, 2020).”
A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques (2021, unknown): “The recently proposed COIL architecture (Gao et al., 2021a) presents an interesting case for this conceptual framework.”
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “Most modern information retrieval systems are based on the index-then-retrieve pipeline where documents are first encoded and stored in an inverted index [10, 13, 15, 26, 34, 55, 58] or a nearest neighbor search index [7, 14, 20, 21, 25, 29, 49, 51] and search results are then constructed based on a”
Condenser: a Pre-training Architecture for Dense Retrieval (2021, methodology): “An alternative is the recently proposed contextualized sparse retrieval model (Gao et al., 2021a).”
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2021, methodology): “, 2021), address vocabulary mismatch (Nogueira and Lin, 2019) and semantic mismatch (Gao et al., 2021), inter alia.”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “Further, COIL [169] utilizes the contextualized token representations of exact matching terms for estimating the relevance.”
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021, methodology): “The sparse retrievers include the traditional re-triever BM25 (Yang et al., 2017) and ﬁve traditional retrievers enhanced by neural networks, including doc2query (Nogueira et al., 2019c), DeepCT (Dai and Callan, 2019), docTTTTT-query (Nogueira et al., 2019a), GAR (Mao et al., 2020), UHD-BERT (Jang e”
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies (2026, unknown): “…et al., 2023; Nogueira et al., 2019) focused on vocabulary expansion to mitigate the document/query vocabulary mismatch problem. uniCOIL (Gao et al., 2021) further expanded on these methods by attempting to reconcile them, learning scalar weights over lexical terms with optional…”
A Survey of Model Architectures in Information Retrieval (2025, unknown): “UNI COIL (Lin and Ma, 2021) further simplifies reduces the dimension of semantic embeddings to 1 — equivalent to learned term weight in learned sparse retrieval models like SPLADE (Formal et al., 2021b,a).”
Late Code Chunking: A Code Chunking Strategy for Repository-Level Code Completion (2026, unknown): “This result empirically validates the importance of exact lexical matching (Gao et al., 2021) in code completion and demonstrates the effectiveness of our Asymmetric Query-Chunk strategy.”