Distilling Knowledge for Fast Retrieval-based Chat-bots
By Amir Vakili Tahami, Kamyar Ghajar, Azadeh Shakery. 2020
32 citations recorded by the source corpus.
Response retrieval is a subset of neural ranking in which a model selects a suitable response from a set of candidates given a conversation history. Retrieval-based chat-bots are typically employed in information seeking conversational systems such as customer support agents. To make pairwise comparisons between a conversation history and a candidate response, two approaches are common: cross-encoders performing full self-attention over the pair and bi-encoders encoding the pair separately. The former gives better prediction quality but is too slow for practical use. In this paper, we propose a new cross-encoder architecture and transfer knowledge from this model to a bi-encoder model using distillation. This effectively boosts bi-encoder performance at no cost during inference time. We perform a detailed analysis of this approach on three response retrieval datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jimmy J. Lin
- Sebastian Hofstätter
- Allan Hanbury
- Sheng-Chieh Lin
- Jheng-Hong Yang
- Mete Sertkan
- Sophia Althammer
- Michael Schröder
In Computer Vision and Pattern Recognition
- Jimmy J. Lin
- Mete Sertkan
- Sebastian Hofstätter
- Allan Hanbury
- Sophia Althammer
- Sheng-Chieh Lin
- Jheng-Hong Yang
- Michael Schröder
Citations in the corpus
- Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, methodology): “We compare our approach with a pointwise MSE loss, defined as follows: This is a standard approach already used by Vakili Tahami et al. [39] and Li et al. [24].”
- Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling (2021, methodology): “Apart from specifically training dense retrieval models, knowledge distillation gains popularity, with general-purpose BERT-style models [17, 33] as well as a range of applications in IR: from sequential recommendation models [35], BERT-based retrieval chatbots [36], BERT-based Question Answering [1”