Anserini

By Peilin Yang, Hui Fang, Jimmy J. Lin. 2018

173 citations recorded by the source corpus.

This work tackles the perennial problem of reproducible baselines in information retrieval research, focusing on bag-of-words ranking models. Although academic information retrieval researchers have a long history of building and sharing systems, they are primarily designed to facilitate the publication of research papers. As such, these systems are often incomplete, inflexible, poorly documented, difficult to use, and slow, particularly in the context of modern web-scale collections. Furthermore, the growing complexity of modern software ecosystems and the resource constraints most academic research groups operate under make maintaining open-source systems a constant struggle. However, except for a small number of companies (mostly commercial web search engines) that deploy custom infrastructure, Lucene has become the de facto platform in industry for building search applications. Lucene has an active developer base, a large audience of users, and diverse capabilities to work with heterogeneous collections at scale. However, it lacks systematic support for ad hoc experimentation using standard test collections. We describe Anserini, an information retrieval toolkit built on Lucene that fills this gap. Our goal is to simplify ad hoc experimentation and allow researchers to easily reproduce results with modern bag-of-words ranking models on diverse test collections. With Anserini, we demonstrate that Lucene provides a suitable framework for supporting information retrieval research. Experiments show that our system efficiently indexes large web collections, provides modern ranking models that are on par with research implementations in terms of effectiveness, and supports low-latency query evaluation to facilitate rapid experimentation

Subfields: Information Systems, Management Science and Operations Research, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Management Science and Operations Research

In Information Retrieval & Search

Citations in the corpus

Passage Re-ranking with BERT (2019, methodology): “com/google-research/bert We use the Anserini toolkit (Yang et al., 2018) to index and retrieve the passages.”
End-to-End Open-Domain Question Answering with BERTserini (2019, background): “Anserini (Yang et al., 2017, 2018) represents recent efforts by researchers to bring academic IR into better alignment with the practice of building real-world search applications, where Lucene has become the de facto platform used in industry.”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “For the passage task, BM25-E is the Anserini (Yang et al., 2018a) system with default parameters.”
Relevance-guided Supervision for OpenQA with ColBERT (2020, methodology): “For BM25 retrieval, we use the Anserini (Yang et al., 2018) toolkit with its default MSMARCO-tuned k1 and b for passage search.”
Document Expansion by Query Prediction (2019, methodology): “We evaluate the following ranking methods: BM25 : We use the Anserini open-source IR toolkit (Yang et al., 2017, 2018) 4 to index the original (non-expanded) documents and BM25 to rank the passages.”
Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations (2021, methodology): “The above result corresponds to an Anserini baseline on the MS MARCO passage leaderboard.”
Supporting Interoperability Between Open-Source Search Engines with the Common Index File Format (2020, methodology): “• Anserini [15] is an IR toolkit built on the popular open-source Lucene search library.”
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages (2022, unknown): “Hybrid = 𝛼 · 𝑠 BM25 + ( 1 − 𝛼 ) · 𝑠 mDPR , where we set 𝛼 = 0 .”
Optimizing Dense Retrieval Model Training with Hard Negatives (2021, methodology): “We list several representative results according to the TREC overview paper [4] and runs onMSMARCO [1] leaderboard, such as BM25 [31], the best traditional retrievalmethod, BERT weighted BM25 (DeepCT) [5].”
Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models (2019, methodology): “e average precision reported in two recent neural IR papers, anonymously referred to as “Paper 1” and “Paper 2”. 3.1 Experimental Setup We began by replicating Lin’s results with the Anserini toolkit [23], using exactly the same experimental settings (tokenization, stemming, etc.) described in an on”
Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-Shot Learning (2019, methodology): “We index the text content of each document using a modified version of Anserini with support for the languages we investigate [36].”
Distant Supervision for Multi-Stage Fine-Tuning in Retrieval-Based Question Answering (2020, methodology): “To make our results comparable to BERTserini, we also use Anserini [27, 28] for retrieval and rank the paragraphs using BM25 with the same parameter settings (k1 = 0.9,b = 4).”
Capreolus: A Toolkit for End-to-End Neural Ad Hoc Retrieval (2020, methodology): “Given a pipeline configured with the above modules, Capreolus performs any needed preprocessing (e.g., creating an Anserini index and determining the searcher’s optimal parameters for the given fold) before beginning a training loop.”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “The other line is a RM3 baseline run with default parameters from the Anserini open-source information retrieval toolkit [66, 67] at AP 0.”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “, 1995) results using the Anserini (Yang et al., 2018a) toolkit.”
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020, methodology): “We ﬁrst apply a preprocessing step to properly index the Wikipedia dump with Lucene using the pyserini library, version 0.9.4.0 (Yang, Fang, and Lin 2017, 2018).”