Applying BERT to Document Retrieval with Birch

By Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, Jimmy J. Lin. 2019

128 citations recorded by the source corpus.

We present Birch, a system that applies BERT to document retrieval via integration with the open-source Anserini information retrieval toolkit to demonstrate end-to-end search over large document collections. Birch implements simple ranking models that achieve state-of-the-art effectiveness on standard TREC newswire and social media test collections. This demonstration focuses on technical challenges in the integration of NLP and IR capabilities, along with the design rationale behind our approach to tightly-coupled integration between Python (to support neural networks) and the Java Virtual Machine (to support document retrieval using the open-source Lucene search library). We demonstrate integration of Birch with an existing search interface as well as interactive notebooks that highlight its capabilities in an easy-to-understand manner.

Subfields: Artificial Intelligence, Information Retrieval & Search, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Large Language Models

Citations in the corpus

How Context Affects Language Models' Factual Predictions (2020, methodology): “Pre-trained language models have also been ﬁne-tuned and adapted to be used as information retrieval systems [Yang et al., 2019b, Yilmaz et al., 2019, Seo et al., 2019].”
Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations (2021, background): “More recently, we recognized that Anserini’s reliance on the Java Virtual Machine (due to Lucene), greatly limited its reach [2, 3], as Python has emerged as the language of choice for both data scientists and researchers.”
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations (2020, background): “5381 Birch [49] - 0.”
Rethinking Query Expansion for BERT Reranking (2020, methodology): “Other existing BERT rerankers use a similar architecture as BERT-MaxP, and apply more complex techniques to improve accuracy, e.g., domain adaptation [2, 16] , fusion with BM25 [16] , and customized pooling layers [11] .”
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval (2019, background): “details about the technical design of our system are presented in Yilmaz et al. (2019), a companion demonstration paper.”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “In this thesis, we extend our own work [73, 72], presenting a novel way to apply BERT to “ad hoc” document retrieval on long documents – particularly, newswire articles – with significant improvements.”