Anserini: Enabling the Use of Lucene for Information Retrieval Research

By Peilin Yang, Hui Fang, Jimmy J. Lin. 2017

423 citations recorded by the source corpus.

Software toolkits play an essential role in information retrieval research. Most open-source toolkits developed by academics are designed to facilitate the evaluation of retrieval models over standard test collections. Efforts are generally directed toward better ranking and less attention is usually given to scalability and other operational considerations. On the other hand, Lucene has become the de facto platform in industry for building search applications (outside a small number of companies that deploy custom infrastructure). Compared to academic IR toolkits, Lucene can handle heterogeneous web collections at scale, but lacks systematic support for evaluation over standard test collections. This paper introduces Anserini, a new information retrieval toolkit that aims to provide the best of both worlds, to better align information retrieval practice and research. Anserini provides wrappers and extensions on top of core Lucene libraries that allow researchers to use more intuitive APIs to accomplish common research tasks. Our initial efforts have focused on three functionalities: scalable, multi-threaded inverted indexing to handle modern web-scale collections, streamlined IR evaluation for ad hoc retrieval on standard test collections, and an extensible architecture for multi-stage ranking. Anserini ships with support for many TREC test collections, providing a convenient way to replicate competitive baselines right out of the box. Experiments verify that our system is both efficient and effective, providing a solid foundation to support future research.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, background): “It has been shown to be robust for both traditional information retrieval tasks, and evidence retrieval for question answering (Yang et al., 2017 BM25 is not trainable, the retrieved evidence considered during ﬁne-tuning is static.”
End-to-End Open-Domain Question Answering with BERTserini (2019, background): “Anserini (Yang et al., 2017, 2018) represents recent efforts by researchers to bring academic IR into better alignment with the practice of building real-world search applications, where Lucene has become the de facto platform used in industry.”
Learning to Retrieve Passages without Supervision (2021, methodology): “For sparse retrieval (i.e. BM25), we utilize the Pyserini library (Lin et al., 2021), built on top of Anserini (Yang et al., 2017, 2018).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “tative BM25 rankings that differ quite a bit in effectiveness. One prominent example appears on the leaderboard of the MS MARCO passage retrieval task: a BM25 ranking produced by the Anserini system [Yang et al., 2017, 2018] scores 0.186 in terms of MRR@10, but the Microsoft BM25 baseline scores two”
Generation-Augmented Retrieval for Open-Domain Question Answering (2020, methodology): “We use Anserini (Yang et al., 2017) for text retrieval of BM25 and G AR with its default parameters.”
Pre-training Tasks for Embedding-based Large-scale Retrieval (2020, methodology): “One example is BM-25 (Robertson et al., 2009), which remains to be the most commonly-used (Nguyen et al., 2016; Yang et al., 2017; 2019a) and hard to beat (Chapelle & Chang, 2011; Lee et al., 2019) algorithm.”
Multi-Stage Document Ranking with BERT (2019, methodology): “We use the Anserini IR toolkit (Yang et al., 2017, 2018), 1 which is built on the popular open-source Lucene search engine.”
Document Expansion by Query Prediction (2019, methodology): “We evaluate the following ranking methods: BM25 : We use the Anserini open-source IR toolkit (Yang et al., 2017, 2018) 4 to index the original (non-expanded) documents and BM25 to rank the passages.”
Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations (2021, methodology): “The above result corresponds to an Anserini baseline on the MS MARCO passage leaderboard.”
Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations (2021, methodology): “The above result corresponds to an Anserini baseline on the MS MARCO passage leaderboard.”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, background): “MSMARCO Dev Natural Questions Test MRR@10 R@50 R@1000 R@5 R@20 R@100 BM25 (anserini) (Yang et al., 2017 (Karpukhin et al., 2020) BERT base - - - - 78.4 85.4 ANCE (single) (Xiong et al., 2020) RoBERTa base 33.0 - 95.9 - 81.9 87.5 ME-BERT (Luan et al., 2020) BERT large 33.8 - - - - - RocketQA ERNIE…”
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020, methodology): “We ﬁrst apply a preprocessing step to properly index the Wikipedia dump with Lucene using the pyserini library, version 0.9.4.0 (Yang, Fang, and Lin 2017, 2018).”
CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning (2021, methodology): “, 2021) to use Pyserini (Yang et al., 2017) with the default parameters k1 = 0.”
SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search (2020, methodology): “for maximizing system recall, which may require special considerations (Grossman et al.,2015). Our work utilizes a variety of existing opensource tools, including OpenNIR (MacAvaney, 2020), Anserini (Yang et al. 2017), and the HuggingFace Transformers library (Wolf et al.,2019). Note on manual vs au”
A Little Bit Is Worse Than None: Ranking with Limited Training Data (2020, neutral): “4 ) as the ﬁrst-stage ranker (Yang et al., 2017).”