Pretrained Transformers for Text Ranking: BERT and Beyond

By Suzan Verberne. 2022

16 citations recorded by the source corpus.

Text ranking takes a central place in Information Retrieval (IR), with Web search as its best-known application. More generally, text ranking models are applicable to any Natural Language Processing (NLP) task in which relevance of information plays a role, from filtering and recommendation applications to question answering and semantic similarity comparisons. Since the rise of BERT in 2019, Transformer models have become the most used and studied architectures in both NLP and IR, and they have been applied to basically any task in our research fields—including text ranking. In a fast-changing research context, it can be challenging to keep lecture materials up to date. Lecturers in NLP are grateful for Dan Jurafsky and James Martin for yearly updating the 3rd edition of their textbook, making Speech and Language Processing the most comprehensive, modern textbook for NLP. The IR field is less fortunate, still relying on older textbooks, extended with a collection of recent materials that address neural models. The textbook Pretrained Transformers for Text Ranking: BERT and Beyond by Jimmy Lin, Rodrigo Nogueira, and Andrew Yates is a great effort to collect the recent developments in the use of Transformers for text ranking. The introduction of the book is well-scoped with clear guidance for the reader about topics that are out of scope (such as user aspects). This is followed by an excellent history section, stating for example:

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jimmy J. Lin
2. Andrew Yates
3. Rodrigo Nogueira
4. Zeynep Akkalyoncu Yilmaz
5. Sebastian Hofstätter
6. Allan Hanbury
7. Zhuyun Dai
8. Jamie Callan

In Computational Theory and Mathematics

1. Rodrigo Nogueira
2. Jimmy J. Lin
3. Andrew Yates

In Large Language Models

1. Jimmy J. Lin
2. Andrew Yates
3. Rodrigo Nogueira
4. Zhuyun Dai
5. Jamie Callan
6. Zeynep Akkalyoncu Yilmaz
7. Sebastian Hofstätter
8. Allan Hanbury

In Natural Language Processing

1. Jimmy J. Lin
2. Rodrigo Nogueira
3. Andrew Yates
4. Zeynep Akkalyoncu Yilmaz
5. Wei Yang
6. Sheng-Chieh Lin
7. Jheng-Hong Yang
8. Haotian Zhang

Citations in the corpus

• Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “This tutorial, which is based on the preprint [21] of a forthcoming book to be published by Morgan and & Claypool under the Synthesis Lectures on Human Language Technologies series, provides an overview of existing work as a single point of entry for practitioners who wish to deploy transformers for”