Zero-shot Neural Retrieval via Domain-targeted Synthetic Query Generation

By Ji Ma, I. Korotkov, Yinfei Yang, Keith B. Hall, Ryan T. McDonald. 2020

32 citations recorded by the source corpus.

Deep neural scoring models have recently been shown to improve ranking quality on a number of benchmarks (Guo et al., 2016; Daiet al., 2018; MacAvaney et al., 2019; Yanget al., 2019a). However, these methods rely on underlying ad-hoc retrieval systems to generate candidates for scoring, which are rarely neural themselves (Zamani et al., 2018). Re-cent work has shown that the performance of ad-hoc neural retrieval systems can be competitive with a number of baselines (Zamani et al.,2018), potentially leading the way to full end-to-end neural retrieval. A major road-block to the adoption of ad-hoc retrieval models is that they require large supervised training sets to surpass classic term-based techniques, which can be developed from raw corpora. Previous work shows weakly supervised data can yield competitive results, e.g., click data (Dehghaniet al., 2017; Borisov et al., 2016). Unfortunately for many domains, even weakly supervised data can be scarce. In this paper, we pro-pose an approach to zero-shot learning (Xianet al., 2018) for ad-hoc retrieval models that relies on synthetic query generation. Crucially, the query generation system is trained on general domain data, but is applied to documents in the targeted domain. This allows us to create arbitrarily large, yet noisy, query-document relevance pairs that are domain targeted. On a number of benchmarks, we show that this is an effective strategy for building neural retrieval models for specialised domains.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (2020, methodology): “All DR baselines use the same BERT-Siamese (base) model as used in various parallel research [9, 12, 13, 15].”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “ team “unique_ptr”. Their approach consisted of generating queries from CORD-19 articles using a model similar to doc2query and then training a reranker on these pseudo query–relevant document pairs [Ma et al., 2020]. The “unique_ptr” team was responsible for the highest “feedback” run (and top-scor”
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering (2022, background): “Previous work (Ma et al., 2020) has explored the importance of the domain adaptation of neural retrievers and highlighted the performance gains in domain-specific retrieval tasks.”
Conformer-Kernel with Query Term Independence for Document Retrieval (2020, methodology): “…alternative to document expansion based methods is to use the neural model to reformulate the query [Nogueira and Cho, 2017, Van Gysel et al., 2017, Ma et al., 2020]—although these approaches have not yet demonstrated retrieval performance that can be considered competitive to other methods…”
Large Dual Encoders Are Generalizable Retrievers (2021, background): “Recent works have shown great improvement under the zero-shot setting for dual encoders by leveraging distillation and synthetic data generation (Thakur et al., 2021; Hofstätter et al., 2021; Ma et al., 2020).”
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale (2020, result): “Similar approaches were also explored in ad-hoc retrieval (Zhang et al., 2020; Ma et al., 2020; MacAvaney et al., 2019).”