EARL: Speedup Transformer-based Rankers with Pre-computed Representation

By Luyu Gao, Zhuyun Dai, Jamie Callan. 2020

9 citations recorded by the source corpus.

Recent innovations in Transformer-based ranking models have advanced the state-of-the-art in information retrieval. However, their performance gains come at a steep computational cost. This paper presents a novel Embed Ahead Rank Later (EARL) framework, which speeds-up Transformer-based rankers by pre-computing representations and keeping online computation shallow. EARL dis-entangles the attention in a typical Transformer-based ranker into three asynchronous tasks and assign each to a dedicated Transformer: query understanding, document understanding, and relevance judging. With such a ranking framework, query and document token representations can be offline computed and reused. We also propose a new judger transformer block that keeps online relevance judging light and shallow. Our experiments demonstrate that EARL can be as effective as previous state-of-the-art BERT rankers in accuracy while substantially faster in evaluation time.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jimmy J. Lin
2. Sebastian Hofstätter
3. Allan Hanbury
4. Andrew Yates
5. Rodrigo Nogueira
6. Sean MacAvaney
7. Nazli Goharian
8. Mete Sertkan

In Information Systems

1. Sebastian Hofstätter
2. Allan Hanbury
3. Rodrigo Nogueira
4. Sean MacAvaney
5. Jimmy J. Lin
6. Nazli Goharian
7. Andrew Yates
8. Kai Hui

In Information Retrieval & Search

1. Jimmy J. Lin
2. Sebastian Hofstätter
3. Allan Hanbury
4. Rodrigo Nogueira
5. Andrew Yates
6. Sean MacAvaney
7. Nazli Goharian
8. Mete Sertkan

In Efficiency & Compression

1. Jimmy J. Lin
2. Allan Hanbury
3. Sebastian Hofstätter
4. Mete Sertkan
5. Rodrigo Nogueira
6. Andrew Yates
7. Arman Cohan
8. Sheng-Chieh Lin

Citations in the corpus

• Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “Evaluation on the MS MARCO passage retrieval test collection and TREC 2019 Deep Learning Track passage retrieval test collection are shown in Table 23, with results copied Gao et al. [2020b]. The six-layer and four-layer student models are shown in row groups (2) and (3), respectively, and the monoB”
• Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, methodology): “Finally, the CLS token output is pooled with single linear layer 𝑊 𝑠 : Concurrently to PreTT, DC-BERT [46] and EARL [12] have been proposed with very similar approaches to split Transformer layers.”
• BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020, background): “Many efforts have also been made to reduce the inference time of transformerbased rankers (Hofstätter et al., 2020a,b; Gao et al., 2020).”
• SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search (2020, background): “…modeling choices (Hofst¨atter et al., 2020; MacAvaney et al., 2020c), caching intermediate representations (Khattab and Zaharia, 2020; MacAvaney et al., 2020b; Gao et al., 2020), or by modifying the index with new terms or weights (Nogueira et al., 2019; Dai and Callan, 2019a; Nogueira, 2019).”