Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval

By Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, Jimmy J. Lin. 2019

170 citations recorded by the source corpus.

This paper applies BERT to ad hoc document retrieval on news articles, which requires addressing two challenges: relevance judgments in existing test collections are typically provided only at the document level, and documents often exceed the length that BERT was designed to handle. Our solution is to aggregate sentence-level evidence to rank documents. Furthermore, we are able to leverage passage-level relevance judgments fortuitously available in other domains to fine-tune BERT models that are able to capture cross-domain notions of relevance, and can be directly used for ranking news articles. Our simple neural ranking models achieve state-of-the-art effectiveness on three standard test collections.

Subfields: Artificial Intelligence, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

Citations in the corpus

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (2020, background): “Indeed, in the span of just a few months, a number of ranking models based on BERT have achieved state-of-the-art results on various retrieval benchmarks [3, 18, 25, 39] and have been proprietarily adapted for deployment by Google1 and Bing2.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “dvance in applying BERT to text ranking. This occurred with three proposed models that were roughly contemporaneous, dating to Spring of 2019, merely a few months after monoBERT: 53 Birch [Akkalyoncu Yilmaz et al., 2019], which was ﬁrst described in Yang et al. [2019e], BERT– MaxP [Dai and Callan, 2”
Document Ranking with a Pretrained Sequence-to-Sequence Model (2020, background): “For Robust04 ( adopt a different architecture than CEDR and Birch, but effectiveness clearly improves as the size of the T5 model increases.”
Relevance-guided Supervision for OpenQA with ColBERT (2020, positive): “We also report strong results when using an out-of-domain transfer learning setting from IR, which work by Akkalyoncu Yilmaz et al. (2019) considers but in the context of neural re-rankers and between standard IR tasks.”
Multi-Stage Document Ranking with BERT (2019, methodology): “Models that can handle longer documents without truncation, such as Yilmaz et al. (2019), should be evaluated on datasets such as the MS MARCO document ranking task.”
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, methodology): “The common way of utilizing the BERT pre-trained Transformer model in a re-ranking scenario [29, 34, 45] is by concatenating query and passage input sequences.”
Local Self-Attention over Long Text for Efficient Document Retrieval (2020, unknown): “1 - bm25_marcomb [25] n/a 0.”
BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020, methodology): “- Birch (Yilmaz et al., 2019) is a re-ranking approach by fine-tuning BERT successively on the MS MARCO and MicroBlog (MB) datasets.”
A Little Bit Is Worse Than None: Ranking with Limited Training Data (2020, methodology): “Alternatively, this obstacle can be entirely avoided with a zero-shot approach: the model is ﬁne-tuned on a passage retrieval dataset and then directly applied to the target corpus (Yilmaz et al., 2019; Nogueira et al., 2020).”
Cross-Lingual Relevance Transfer for Document Retrieval (2019, methodology): “cross-validation. All candidate documents are resorted by the above score S doc, which serves as the ﬁnal output. Our approach is a straightforward adaptation of the evidence combination technique of Yilmaz et al. (2019), except using mBERT. To be precise, we apply an mBERT model that has been ﬁne-t”
Information Retrieval: A Biomedical and Health Perspective (2020, background): “The paper that the editorial described had discovered that the lack of expression of a gene (CDX2) was associated with benefit from adjuvant chemotherapy [217].”
Automated Fact-Checking for Assisting Human Fact-Checkers (2021, background): “Whereas most models intend to assess the relevance of full documents against the query, recent efforts try to capture relevance also at the sentence level and combine it to score full documents [Akkalyoncu Yilmaz et al., 2019].”
SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search (2020, background): “These models also facilitate effective relevance signal transfer; Yilmaz et al. (2019) demonstrate that the relevance signals learned from BERT can easily transfer across collections (reducing the chance of overﬁtting a particular collection).”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “In this thesis, we extend our own work [73, 72], presenting a novel way to apply BERT to “ad hoc” document retrieval on long documents – particularly, newswire articles – with significant improvements.”
Early Exiting BERT for Efficient Document Ranking (2020, background): “…recent years, large-scale pre-trained language models, especially those based on the transformer architecture (Vaswani et al., 2017), have been applied to IR tasks and have pushed the state of the art even further (Nogueira and Cho, 2019; Dai and Callan, 2019; Yilmaz et al., 2019; Li et al., 2020).”
MarkedBERT: Integrating Traditional IR Cues in Pre-trained Language Models for Passage Retrieval (2020, background): “The same trend is witnessed in the IR community, where several works emerged applying these models to IR tasks such as document retrieval [1, 5, 9], passage ranking [8], and question answering [17].”
Applying BERT to Document Retrieval with Birch (2019, methodology): “Birch allows the community to replicate the state-of-the-art document ranking results presented in Yilmaz et al. (2019) and Yang et al. (2019c).”