TILDE: Term Independent Likelihood moDEl for Passage Re-ranking

By Shengyao Zhuang, Guido Zuccon. 2021

92 citations recorded by the source corpus.

Deep language models (deep LMs) are increasingly being used for full text retrieval or within cascade retrieval pipelines as later-stage re-rankers. A problem with using deep LMs is that, at query time, a slow inference step needs to be performed -- this hinders the practical adoption of these powerful retrieval models, or limits sensibly how many documents can be considered for re-ranking. We propose the novel, BERT-based, Term Independent Likelihood moDEl (TILDE), which ranks documents by both query and document likelihood. At query time, our model does not require the inference step of deep language models based retrieval approaches, thus providing consistent time-savings, as the prediction of query terms' likelihood can be pre-computed and stored during index creation. This is achieved by relaxing the term dependence assumption made by the deep LMs. In addition, we have devised a novel bi-directional training loss which allows TILDE to maximise both query and document likelihood at the same time during training. At query time, TILDE can rely on its query likelihood component (TILDE-QL) solely, or the combination of TILDE-QL and its document likelihood component (TILDE-DL), thus providing a flexible trade-off between efficiency and effectiveness. Exploiting both components provide the highest effectiveness at a higher computational cost while relying only on TILDE-QL trades off effectiveness for faster response time due to no inference being required. TILDE is evaluated on the MS MARCO and TREC Deep Learning 2019 and 2020 passage ranking datasets. Empirical results show that, compared to other approaches that aim to make deep language models viable operationally, TILDE achieves competitive effectiveness coupled with low query latency.

Subfields: Artificial Intelligence, Management Science and Operations Research, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jian Pei
2. Shengyao Zhuang
3. Guido Zuccon
4. Houxing Ren
5. Linjun Shou
6. Ming Gong
7. Daxin Jiang
8. Jiawei Han

In Management Science and Operations Research

1. Jian Pei
2. Shengyao Zhuang
3. Guido Zuccon
4. Linjun Shou
5. Ming Gong
6. Daxin Jiang

In Information Retrieval & Search

1. Jian Pei
2. Shengyao Zhuang
3. Guido Zuccon
4. Houxing Ren
5. Linjun Shou
6. Ming Gong
7. Daxin Jiang

Citations in the corpus

• Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “In other words: TILDE generates query tokens, not queries (i.e. sequences of tokens) – and these then are not representative of the inputs that the model will observe at retrieval time (e.g., Table 4 shows queries in the two datasets considered in our work consists, on average, of 12.07 and 14.8 que”