Expansion via Prediction of Importance with Contextualization

By Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, Ophir Frieder. 2020

122 citations recorded by the source corpus.

The identification of relevance with little textual context is a primary challenge in passage retrieval. We address this problem with a representation-based ranking approach that: (1) explicitly models the importance of each term using a contextualized language model; (2) performs passage expansion by propagating the importance to similar terms; and (3) grounds the representations in the lexicon, making them interpretable. Passage representations can be pre-computed at index time to reduce query-time latency. We call our approach EPIC (Expansion via Prediction of Importance with Contextualization). We show that EPIC significantly outperforms prior importance-modeling and document expansion approaches. We also observe that the performance is additive with the current leading first-stage retrieval methods, further narrowing the gap between inexpensive and cost-prohibitive passage ranking approaches. Specifically, EPIC achieves a MRR@10 of 0.304 on the MS-MARCO passage ranking dataset with 78ms average query latency on commodity hardware. We also find that the latency is further reduced to 68ms by pruning document representations, with virtually no difference in effectiveness.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval (2021, methodology): “A second solution to this problem, that has been chosen by recent works [1, 8, 17, 33], is to estimate the importance of each term of the vocabulary implied by each term of the document (or query), i.e. to compute an interaction matrix between the document or query tokens and all the tokens from…”
Conformer-Kernel with Query Term Independence for Document Retrieval (2020, result): “Several recent neural IR models [Mitra et al., 2019, Dai and Callan, 2019b,a, Mackenzie et al., 2020, Dai and Callan, MacAvaney et al., 2020] that incorporate the QTI assumption have obtained promising results under the full retrieval setting.”
SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (2021, methodology): “The above equation can be seen as a form of query/document expansion, as observed in [1, 16], since for each token of the vocabulary the model predicts a new weight w j .”
SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search (2020, background): “…more efﬁcient modeling choices (Hofst¨atter et al., 2020; MacAvaney et al., 2020c), caching intermediate representations (Khattab and Zaharia, 2020; MacAvaney et al., 2020b; Gao et al., 2020), or by modifying the index with new terms or weights (Nogueira et al., 2019; Dai and Callan, 2019a;…”
TILDE: Term Independent Likelihood moDEl for Passage Re-ranking (2021, unknown): “These deep LMs have been exploited to improve search along four main patterns, which are characterised by a trade-off between effectiveness and efficiency, exposed in the high-level comparison of • representation based methods , like EPIC [16], ColBERT [12], RepBERT [31], CLEAR [10] and ANCE [27].”
Learning Passage Impacts for Inverted Indexes (2021, methodology): “Table 3 shows the effect of re-ranking the top 1000 candidates produced by DeepImpact using two complex re-rankers, EPIC and ColBERT.”