Contextual Document Embeddings

By John X. Morris, Alexander M. Rush. 2024

Subfields: Information Retrieval & Search, Representation & Self-Supervised Learning

Citations in the corpus

• NEAR: Neighborhood-Aware Residuals for Memory Retrieval (2026, unknown): “A common approach is embedding-based retrieval, which encodes queries and memories separately and ranks memories by vector similarity (Morris & Rush, 2025).”
• Zero-Shot Contextual Embeddings via Offline Synthetic Corpus Generation (2025, unknown): “Across the MTEB benchmark, ZEST using its exemplar-guided synthetic context achieves performance strikingly close to the CDE model that leverages full target corpus access, with an average difference of merely 0.29 NDCG@10 points.”
• Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings (2025, unknown): “Similarly, Morris and Rush (2024) demonstrate that appending learned "corpus" embeddings to queries and documents can further improve retrieval.”
• DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers (2025, unknown): “CDE (Morris and Rush, 2025) further proposes a two-stage model architecture that integrates corpus-level information into document embeddings.”
• LLMs Are Also Effective Embedding Models: An In-depth Overview (2024, unknown): “[106] designed the alternative contrastive learning objective and the new contextual architecture to incorporate neighbor document information into document embeddings.”