Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations

By Jimmy J. Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, Rodrigo Nogueira. 2021

105 citations recorded by the source corpus.

Pyserini is an easy-to-use Python toolkit that supports replicable IR research by providing effective first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. We also describe how our group has built a culture of replicability through shared norms and tools that enable rigorous automated testing.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

Improving Wikipedia verifiability with AI (2022, background): “…et al., 2019; Mao et al., 2020) to translate the citation context into query text , and then matches the resulting query—a sparse bag-of-words vector—on a BM25 index (Robertson et al., 1995; Baeza-Yates et al., 1999; Manning et al., 2008; Robertson & Zaragoza, 2009; Lin et al., 2021) of Sphere.”
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus (2021, methodology): “For sparse methods, this goal is typically achieved with an inverted index—a space-efﬁcient technique entertaining the support of multiple robust libraries such as Pyserini (Lin et al., 2021b).”
Learning to Retrieve Passages without Supervision (2021, methodology): “For sparse retrieval (i.e. BM25), we utilize the Pyserini library (Lin et al., 2021), built on top of Anserini (Yang et al., 2017, 2018).”
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, methodology): “We use Pyserini [28] for the search engine, where indices are built ofﬂine using a BM25 sparse retriever with text for each product concatenated from the title, description, overview, and customization options.”
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval (2021, methodology): “We were not able to ﬁnd a BM25 implementation that supports Telugu. the target languages, we use a BM25-based lexical retriever implementation by Pyserini (Lin et al., 2021).”
A Replication Study of Dense Passage Retriever (2021, methodology): “Code associated with our retrieval experiments is packaged in our Pyserini IR toolkit2 (Lin et al., 2021) and code associated with our end-to-end QA experiments is part of our PyGaggle toolkit3 for neural text ranking.”
Simple Entity-Centric Questions Challenge Dense Retrievers (2021, methodology): “Experimental settings of BM25 In our experiments, we use the Pyserini (Lin et al., 2021) implementation of unigram BM25 with default parameters.”