Reranking for Efficient Transformer-based Answer Selection
By Yoshitomo Matsubara, Thuy Vu, Alessandro Moschitti. 2020
48 citations recorded by the source corpus.
IR-based Question Answering (QA) systems typically use a sentence selector to extract the answer from retrieved documents. Recent studies have shown that powerful neural models based on the Transformer can provide an accurate solution to Answer Sentence Selection (AS2). Unfortunately, their computation cost prevents their use in real-world applications. In this paper, we show that standard and efficient neural rerankers can be used to reduce the amount of sentence candidates fed to Transformer models without hurting Accuracy, thus improving efficiency up to four times. This is an important finding as the internal representation of shallower neural models is dramatically different from the one used by a Transformer model, e.g., word vs. contextual embeddings.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Luca Soldaini
- Yoshitomo Matsubara
- Marco Levorato
- Francesco Restuccia
- Alessandro Moschitti
- Aliaksei Severyn
- Danilo Croce
- Kateryna Tymoshenko
Citations in the corpus
- Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges (2021, background): “…on text ranking, specifically answer sentence selection tasks with question answering datasets, Soldaini and Moschitti [132] add classification layers to intermediate stages of RoBERTa to build sequential (neural) rerankers [95] inside as early exits, and propose the Cascade Transformer models.”
- The Cascade Transformer: an Application for Efficient Answer Sentence Selection (2020, methodology): “Reranking in QA and IR The approach introduced in this paper is inspired by our previous work (Matsubara et al., 2020); there, we used a fast AS2 neural model to select a subset of instances to be input to a transformer model.”