Efficiency Implications of Term Weighting for Passage Retrieval

By Joel M. Mackenzie, Zhuyun Dai, Luke Gallagher, Jamie Callan. 2020

28 citations recorded by the source corpus.

Language model pre-training has spurred a great deal of attention for tasks involving natural language understanding, and has been successfully applied to many downstream tasks with impressive results. Within information retrieval, many of these solutions are too costly to stand on their own, requiring multi-stage ranking architectures. Recent work has begun to consider how to "backport" salient aspects of these computationally expensive models to previous stages of the retrieval pipeline. One such instance is DeepCT, which uses BERT to re-weight term importance in a given context at the passage level. This process, which is computed offline, results in an augmented inverted index with re-weighted term frequency values. In this work, we conduct an investigation of query processing efficiency over DeepCT indexes. Using a number of candidate generation algorithms, we reveal how term re-weighting can impact query processing latency, and explore how DeepCT can be used as a static index pruning technique to accelerate query processing without harming search effectiveness.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, methodology): “The same principles that applied to traditional IR systems to achieve low query latency also apply to novel neural ranking models: We need to transfer as much computation and data transformation to the indexing phase as possible to require less resources at query time [31, 32].”
Learning Passage Impacts for Inverted Indexes (2021, methodology): “For DeepCT , we used the source code and data provided by Mackenzie et al. [15].”
Conformer-Kernel with Query Term Independence for Document Retrieval (2020, result): “Several recent neural IR models [Mitra et al., 2019, Dai and Callan, 2019b,a, Mackenzie et al., 2020, Dai and Callan, MacAvaney et al., 2020] that incorporate the QTI assumption have obtained promising results under the full retrieval setting.”