A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models
By Oleg Lesota, Navid Rekabsaz, Daniel Cohen, Klaus Antonius Grasserbauer, Carsten Eickhoff, Markus Schedl. 2021
21 citations recorded by the source corpus.
Existing neural ranking models follow the text matching paradigm, where document-to-query relevance is estimated through predicting the matching score. Drawing from the rich literature of classical generative retrieval models, we introduce and formalize the paradigm of deep generative retrieval models defined via the cumulative probabilities of generating query terms. This paradigm offers a grounded probabilistic view on relevance estimation while still enabling the use of modern neural architectures. In contrast to the matching paradigm, the probabilistic nature of generative rankers readily offers a fine-grained measure of uncertainty. We adopt several current neural generative models in our framework and introduce a novel generative ranker T-PGN, which combines the encoding capacity of Transformers with the Pointer Generator Network model. We conduct an extensive set of evaluation experiments on passage retrieval, leveraging the MS MARCO Passage Re-ranking and TREC Deep Learning 2019 Passage Re-ranking collections. Our results show the significantly higher performance of the T-PGN model when compared with other generative models. Lastly, we demonstrate that exploiting the uncertainty information of deep generative rankers opens new perspectives to query/collection understanding, and significantly improves the cut-off prediction task.
Subfields: Information Retrieval & Search
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Retrieval & Search
- Sebastian Riedel
- Fabio Petroni
- Wen-tau Yih
- Patrick Lewis
- Meredith Ringel Morris
- Adam Roberts
- Lora Aroyo
- Noam Shazeer
Citations in the corpus
- LaMDA: Language Models for Dialog Applications (2022, background): “Recent work in this direction has expanded and elaborated on neural models’ ability to retrieve and rank passages [40].”
- Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…strand of research is that of query likelihood models, which, in their latest incarnations, use autoregressive models to (re)rank passages according to the probability P ( q | p ) of a query q given the passage p (Nogueira dos Santos et al., 2020; Zhuang and Zuccon, 2021; Lesota et al., 2021).”