Precise Zero-Shot Dense Retrieval without Relevance Labels

By Luyu Gao, Xueguang Ma, Jimmy J. Lin, Jamie Callan. Annual Meeting of the Association for Computational Linguistics, 2022

788 citations recorded by the source corpus.

While dense retrieval has been shown to be effective and efficient across tasks and languages, it remains difficult to create effective fully zero-shot dense retrieval systems when no relevance labels are available. In this paper, we recognize the difficulty of zero-shot learning and encoding relevance. Instead, we propose to pivot through Hypothetical Document Embeddings (HyDE). Given a query, HyDE first zero-shot prompts an instruction-following language model (e.g., InstructGPT) to generate a hypothetical document. The document captures relevance patterns but is “fake” and may contain hallucinations. Then, an unsupervised contrastively learned encoder (e.g., Contriever) encodes the document into an embedding vector. This vector identifies a neighborhood in the corpus embedding space, from which similar real documents are retrieved based on vector similarity. This second step grounds the generated document to the actual corpus, with the encoder’s dense bottleneck filtering out the hallucinations. Our experiments show that HyDE significantly outperforms the state-of-the-art unsupervised dense retriever Contriever and shows strong performance comparable to fine-tuned retrievers across various tasks (e.g. web search, QA, fact verification) and in non-English languages (e.g., sw, ko, ja, bn).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5c32c653735b43a0a8923ca65ac191bd4bf15311

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Jamie Callan
3. Luyu Gao
4. Graham Neubig
5. Jimmy J. Lin
6. Minlie Huang
7. Weizhu Chen
8. Mohit Bansal

In Computer Vision and Pattern Recognition

1. Jamie Callan
2. Luyu Gao
3. Jianfeng Gao
4. Graham Neubig
5. Weizhu Chen
6. Nan Duan
7. Yelong Shen
8. Frank F. Xu

Citations in the corpus

• Active Retrieval Augmented Generation (2023, methodology): “This shares a similar spirit with methods that use generated hypothetical titles or paragraphs from LMs as retrieval queries or evidences (Gao et al., 2022; Sun et al., 2022; Yu et al., 2022; Mao et al., 2021).”
• Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, neutral): “Query2doc [168] and HyDE [169] use the original query to generate a pseudo document, which is later used as the query for retrieval.”
• LightRAG: Simple and Fast Retrieval-Augmented Generation (2024, positive): “By accessing both specific details and overarching themes, LightRAG adeptly responds to complex queries involving interconnected topics, providing contex-tually relevant answers of high quality.”
• Retrieval-Augmented Generation for Large Language Models: A Survey (2023, methodology): “HyDE constructs query vectors using textual cues to generate a “hypothetical” document capturing essential patterns [ Gao et al. , 2022 ] .”
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting (2023, methodology): “HyDE (Gao et al., 2022) uses LLMs to augment queries by generating hypothetical documents for unsupervised retrieval.”
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models (2024, neutral): “In this part, we will briefly review studies with these two types of generators: parameter-accessible (white-box) and parameter-inaccessible (black-box).”
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023, methodology): “To alleviate these issues, there is a growing trend of augmenting LLMs with tools (Mialon et al., 2023; Gou et al., 2023), e.g., a code interpreter (Gao et al., 2022b; Shao et al., 2023) or a search engine (Nakano et al., 2021), in an attempt to offload sub-tasks to more qualified experts, or to…”
• Large Language Models for Information Retrieval: A Survey (2023, neutral): “In this section, we will introduce these two methods in detail.”
• A Systematic Multi-Domain Evaluation of Document Retrievers (2026, unknown): “Retrievers. docT5query [58] expands documents by generating multiple queries for each and retrieving with BM25. query2doc [80] instead expands the query, generating a pseudo-document and again retrieving with BM25, while HyDE [19] generates a hypothetical document and retrieves with the Contriever.”
• BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment (2026, unknown): “Raw q t (no rewrite) 25.4 HyDE [13] 11.2 Query2Doc [41] 14.5 FORGE-RL (ours) 29.7 specific retrieval rewards is a more efficient training signal than scale alone.”
• MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL (2026, unknown): “HyDE [11] generates hypothetical documents from the query for zero-shot dense retrieval, while Query2Doc [33] expands queries with pseudo-documents through few-shot prompting.”
• flexvec: SQL Vector Retrieval with Programmatic Embedding Modulation (2026, unknown): “Recent RAG pipelines frequently incorporate LLM-based hypothetical document generation (e.g., HyDE [10]), late interaction models (ColBERT [11]), or neural rerankers applied after initial retrieval.”
• MDKeyChunker: What Does One LLM Call per Chunk Buy for Markdown Retrieval? (2026, unknown): “HyDE [Gao et al., 2023] works at query time instead, generating a hypothetical document for each query; the questions in MDKeyChunker are produced at index time and are closer to doc2query.”
• Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation (2026, unknown): “Gao et al. (2023) introduced HyDE to bridge the semantic gap in zero-shot retrieval by generating hypothetical documents from queries and us-ing their embeddings for search.”
• Out of Style: RAG's Fragility to Linguistic Variation (2025, unknown): “A.1 Linguistic Variations In this section, we detail our design of linguistic variations as well as our qualitative analysis of the queries we generated.”
• InteractiveSurvey: An LLM-based Personalized and Interactive Survey Paper Generation System (2025, unknown): “To facilitate accurate retrieval, we employ Hypothetical Document Embeddings (HyDE, Gao et al. (2023a)), which generates potential pseudo-descriptions to the categorization criterion k as queries for semantic matching.”
• ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025, unknown): “These methods prompt LLMs to generate query-related documents (Wang et al., 2023a; Jagerman et al., 2023; Gao et al., 2023), leverage Chain-of-Thought (CoT) reasoning results (Wei et al., 2022b; Trivedi et al., 2023), or utilize specific keywords (Li et al., 2024; Jagerman et al., 2023) to expand…”