Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures

By Sebastian Bruch, Jimmy J. Lin. 2013

110 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Document Ranking with a Pretrained Sequence-to-Sequence Model (2020, methodology): “This gives rise to the standard multi-stage pipeline architecture of keyword retrieval followed by reranking using one or more machine learning models [1, 10].”
Multi-Stage Document Ranking with BERT (2019, background): “This basic approach has received much interest in academia (Matveeva et al., 2006; Wang et al., 2011; Asadi and Lin, 2013; Chen et al., 2017; Mackenzie et al., 2018) as well as industry.”
Anserini (2018, background): “Although we have not explicitly addressed this (next) challenge, we believe that baseline bag-of-words ranking models also have an important role to play in modern multi-stage ranking architectures, which have been explored by researchers [8, 16, 17, 36, 49, 55] and deployed commercially [33, 43].”
Anserini: Enabling the Use of Lucene for Information Retrieval Research (2017, background): “In particular, we advocate a multi-stage ranking architecture [3, 6, 14, 17] so that researchers will not need to directly work with native Lucene scoring APIs.”
Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations (2021, background): “In particular, the importance of stable first-stage retrieval within a multi-stage ranking architecture [5, 25, 34] has become even more important, as it provides the foundation for increasingly-complex modern approaches that leverage hybrid techniques.”
Query Driven Algorithm Selection in Early Stage Retrieval (2017, background): “Multi-stage retrieval has become the dominant model in modern web search systems [3, 4, 9, 36, 37, 40].”
Quality versus efficiency in document scoring with learning-to-rank models (2016, unknown): “Asadi and Lin (2013b) ; Asadi, Lin, and de Vries (2013) also propose runtime optimizations for tree-based machine learning models.”
Anytime Ranking for Impact-Ordered Indexes (2015, background): “In a multi-stage retrieval architecture where the initial ranking serves as input to machine-learned ranking models [4], this might yield the same level of effectiveness with less overall computational effort.”
An Exploration of Approaches to Integrating Neural Reranking Models in Multi-Stage Ranking Architectures (2017, background): “Fundamentally, neural network models (and even more broadly, learning-to-rank approaches) for a variety of information retrieval tasks behave as rerankers in a multi-stage ranking architecture [1, 5, 8, 11, 14, 18, 22].”
Which BM25 Do You Mean? A Large-Scale Reproducibility Study of Scoring Variants (2020, methodology): “Although learning-to-rank approaches and neural ranking models are widely used today, they are typically deployed as part of a multi-stage reranking architecture, over candidate documents supplied by a simple term-matching method using traditional inverted indexes [1].”
A Comparison of Document-at-a-Time and Score-at-a-Time Query Evaluation (2017, background): “For multi-stage ranking algorithms [4, 36] that may require large values of k, JASS may be a good choice.”
Old dogs are great at new tricks: column stores for ir prototyping (2014, methodology): “Our experiments focused on conjunctive query evaluation, where the document must contain all query terms; previous work [1] has shown that this approach yields comparable end-to-end eﬀectiveness to disjunctive query evaluation, but is faster.”
The Cascade Transformer: an Application for Efficient Answer Sentence Selection (2020, background): “Asadi and Lin (2013) studied effectiveness/efﬁciency trade-offs with three candidate generation approaches.”
Approximate Nearest Neighbor Search and Lightweight Dense Vector Reranking in Multi-Stage Retrieval Architectures (2020, background): “Multi-stage pipeline architectures are widely adopted for building retrieval systems, both in real-world deployments [16] as well as by researchers [1].”
Reranking for Efficient Transformer-based Answer Selection (2020, background): “Asadi and Lin [1] studied effectiveness/efficiency trade-offs with three candidate selection approaches.”
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset (2020, background): “Multi-stage search architectures represent the most common design for modern search engines, with work in academia dating back over a decade (Matveeva et al., 2006; Wang et al., 2011; Asadi and Lin, 2013).”
Applying BERT to Document Retrieval with Birch (2019, background): “, BM25) over a standard inverted index, followed by a reranker that rescores the candidate list of documents (Asadi and Lin, 2013).”
Assessing efficiency–effectiveness tradeoffs in multi-stage retrieval systems without using relevance judgments (2015, methodology): “Asadi and Lin conclude that conjunctive WAND was the best compromise as the final effectiveness results were statistically indistinguishable from the disjunctive WAND filter.”