IDST at TREC 2019 Deep Learning Track: Deep Cascade Ranking with Generation-based Document Expansion and Pre-trained Language Modeling

By Ming Yan, Chenliang Li, Chen Wu, Jiangnan Xia, Wei Wang. 2019

40 citations recorded by the source corpus.

Subfields: Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Model Architectures

Citations in the corpus

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (2020, background): “Random negatives (BM25 + Rand Neg) [9, 15]. Participants in TREC DL found the passage training labels cleaner than the post-constructed document labels and lead to better results on the document task [31]. Recent DR research also ﬁnds it helps training convergence to include BM25 Negatives to provid”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “–T5 0.642 0.462 0.802 (2a) BM25 + RM3 0.518 0.427 0.788 (2b) w/ doc2query–base 0.564 0.438 0.777 (2c) w/ doc2query–T5 0.655 0.525 0.855 (3) BM25 + RM3 + monoBERTLarge 0.742 0.529 0.788 (4) TREC Best [Yan et al., 2019] 0.765 0.503 - Table 16: The effectiveness of doc2query on the TREC 2019 Deep Learn”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “4) IDST is a two-stage cascade ranking pipeline proposed by Yan et al. (2020) which used both document expansion and crossattention ensemble reranking with tailored BERT model pre-training.”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “Such a strategy has been commonly adopted in practical systems (Yan et al., 2019; Han et al., 2020).”
Optimizing Dense Retrieval Model Training with Hard Negatives (2021, background): “The trained models are directly evaluated on the document task because this produces better retrieval performance [29, 30].”
Conformer-Kernel with Query Term Independence for Document Retrieval (2020, background): “Several of these approaches—e.g., [Yilmaz et al., 2019, Yan et al., 2019]—employ BERT [Devlin et al., 2018], with large-scale pretraining, as their core architecture.”
Long Document Ranking with Query-Directed Sparse Transformer (2020, methodology): “This leads to various retrofitted solutions that adapted the document ranking tasks to standard BERT which takes at most 512 tokens per sequence (Dai and Callan, 2019; Yang et al., 2019; Yan et al., 2020; Nogueira et al., 2019).”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “To implement the reranker, a typical approach is to employ the cross-encoder as the ranking model, showing substantial improvements over the traditional methods [193], [272]–[274].”
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021, methodology): “…a retriever, PLM-based rerankers with the cross-encoder architecture have recently been applied on passage re-ranking to improve the retrieval results (Qiao et al., 2019; Nogueira and Cho, 2019; Wang et al., 2019; Yan et al., 2019), and yield substantial improvements over the traditional methods.”
Local Self-Attention over Long Text for Efficient Document Retrieval (2020, unknown): “More recently, Transformer [7] based architectures have been employed to learn contextual representations which have led to bigger improvements [11, 18, 19, 24].”