Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations

By Jimmy J. Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, Rodrigo Nogueira, David R. Cheriton. 2021

673 citations recorded by the source corpus.

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.

Subfields: Artificial Intelligence, Materials Chemistry, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Information Retrieval & Search

Citations in the corpus

Improving Wikipedia verifiability with AI (2022, background): “For example, veriﬁcation requires the ability to detect logical entailment in natural language and to convert claims and their context to the best search term for ﬁnding evidence—two long-standing problems that have been primarily investigated in somewhat synthetic settings (Bowman et al., 2015;…”
The ROOTS Search Tool: Data Transparency for LLMs (2023, methodology): “We index respective subsets of the corpus using Pyserini (Lin et al., 2021), a leading toolkit for reproducible IR research.”
Are We Done with MMLU? (2024, methodology): “We use BM25 to retrieve relevant paragraphs from enwiki-paragraphs and msmarco-v1-passage corpus provided by Pyserini [17].”
Rethinking with Retrieval: Faithful Large Language Model Inference (2022, methodology): “In particular, we use the re-implementation of the sparse retrieval BM25 4 in Karpukhin et al. (2020) from Pyserini (Lin et al., 2021).”
Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction (2022, methodology): “As baseline, we utilize BM25 as implemented by Pyserini [29].”
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (2024, positive): “Following the official benchmark, we evaluate our method using Pyserini (Lin et al., 2021), and use nDCG@10 as the primary evaluation metric (Recall@100 is also measured and reported in Appendix C.1).”
Improving Passage Retrieval with Zero-Shot Question Generation (2022, methodology): “For BM25, we use the precomputed top-k passages outputs from the pyserini toolkit (Lin et al., 2021).”
Overview of Touché 2022: Argument Retrieval (2022, positive): “23 Initial retrieval of top-40 documents was performed with a Pyserini [49] implementation of BM25F with default parameters ( k 1 = 0 .”
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “We also use the Pyserini implementation for this baseline.”
In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval (2021, methodology): “To characterize end-to-end effectiveness and efﬁciency, we perform sparse retrieval with the Py-serini toolkit (Lin et al., 2021) and dense retrieval with Faiss (Johnson et al., 2017), but implement the score combination in separate custom code.”
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore (2023, positive): “We use BM25 from Pyserini (Lin et al., 2021).”
Overview of the TREC 2021 Deep Learning Track (2025, neutral): “In the reranking subtask, participants were provided with an initial ranking of 100 documents, which was retrieved using Pyserini [Lin et al., 2021b].”
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2021, methodology): “We use Pyserini (Lin et al., 2021a) for all sparse models used in this work including BM25 and UniCOIL.”
Benchmarking Retrieval-Augmented Generation for Medicine (2024, methodology): “In M ED R AG , BM25 is implemented with Pyserini (Lin et al., 2021) 20 using the default hyperparameters to index snippets from all corpora.”
Precise Zero-Shot Dense Retrieval without Relevance Labels (2022, positive): “A variety of meth-ods such as negative mining (Xiong et al., 2021; Qu et al., 2021), distillation (Qu et al., 2021; Lin et al., 2021b; Hofstätter et al., 2021) and task-specific pre-training (Izacard et al., 2021; Gao and Callan, 2021; Lu et al., 2021; Gao and Callan, 2022; Liu and Shao, 2022) have…”
TU Wien at TREC DL and Podcast 2021: Simple Compression for Dense Retrieval (2021, methodology): “Then we apply weighted interpolation on the individual results as described and implemented in Pyserini [11].”