Document Expansion by Query Prediction

By Rodrigo Nogueira, Wei Yang, Jimmy J. Lin, Kyunghyun Cho. 2019

528 citations recorded by the source corpus.

One technique to improve the retrieval effectiveness of a search engine is to expand documents with terms that are related or representative of the documents' content.From the perspective of a question answering system, this might comprise questions the document can potentially answer. Following this observation, we propose a simple method that predicts which queries will be issued for a given document and then expands it with those predictions with a vanilla sequence-to-sequence model, trained using datasets consisting of pairs of query and relevant documents. By combining our method with a highly-effective re-ranking component, we achieve the state of the art in two retrieval tasks. In a latency-critical regime, retrieval results alone (without re-ranking) approach the effectiveness of more computationally expensive neural re-rankers but are much faster.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

Citations in the corpus

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (2020, methodology): “For the latency of Anserini’s BM25, doc2query, and docTTTT-query, we use the authors’ [26, 28] Anserini-based implementation.”
Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “For example, one can augment stored documents by generating possible queries that might be answered by them (Nogueira et al., 2019; Nogueira and Lin, 2021).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “architectures, researchers have proposed additional innovations, including document expansion [32, 34] and term importance pre-”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, background): “models before indexing (Nogueira et al., 2019b) or learn contextual term weights (Dai and Callan, 2020).”
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021, methodology): “…has been used for various purposes, such as data augmentation (Alberti et al., 2019; Lewis et al., 2019b; Lee et al., 2021), improved retrieval (Nogueira et al., 2019), generative modelling for contextual QA (Lewis and Fan, 2018), as well as being studied in its own right (Du et al., 2017;…”
Document Ranking with a Pretrained Sequence-to-Sequence Model (2020, background): “on top of BM25+RM3 because RM3 is known to reduce effectiveness when evaluated using these relevance judgments (Nogueira et al., 2019b).”
Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval (2019, methodology): “The experimental settings mainly follow the methodology used in previous neural passage ranking/re-ranking studies [22, 23] MS MARCO [21] is a question-to-passage retrieval dataset with 8 .”
Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations (2021, methodology): “Pyserini also provides document expansion baselines using our doc2query method [21]; the latest model uses T5 [24] as described in Nogueira and Lin [19].”
Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations (2021, methodology): “also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “Recently, researchers have utilized deep learning to improve traditional passage retrievers, including document expansions (Nogueira et al., 2019c), question expansions (Mao et al., 2020) and term weight estimation (Dai and Callan, 2019).”
Context-Aware Document Term Weighting for Ad-Hoc Search (2020, methodology): “In terms of efficiency, re-ranking models like BERT-FirstP are computationally expensive at the query time [27, 29].”
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations (2020, methodology): “m embeddings is required (rather than only computing similarity scores between the query and document). Within the realm of transformer-based models for ad-hoc ranking, to our knowledge only [27] and [32] acknowledge that retrieval speed is substantially impacted by using a deep transformer network.”
Selective Weak Supervision for Neural Information Retrieval (2020, methodology): “The MS MARCO ranking task is a passage ranking for more natural language queries.”
SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (2021, methodology): “A first solution to this problem consists in expanding documents using generative approaches such as doc2query [19] and docTTTTTquery [18] to predict expansion words for documents.”
SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval (2020, methodology): “Focusing on the relevant relationship between a passage/document and corresponding query, DeepCT [2] and Doc2Query [11] learn PLM-based models to enhance the performance of traditional BoW meth-ods.”
Zero-shot Neural Retrieval via Domain-targeted Synthetic Query Generation (2020, methodology): “Nogueira et al. (2019) employed query generation in the context of information retrieval.”
Efficiency Implications of Term Weighting for Passage Retrieval (2020, background): “However, approaches that augment an inverted index to improve search effectiveness are yet to be scrutinized with respect to search efficiency beyond some simple measures such as inference throughput [20] or mean query latency [19].”