Nomic Embed: Training a Reproducible Long Context Text Embedder

By Zach Nussbaum, John X. Morris, Brandon Duderstadt, Andriy Mulyar. Trans. Mach. Learn. Res., 2024

418 citations recorded by the source corpus.

This technical report describes the training of nomic-embed-text-v1, the first fully reproducible, open-source, open-weights, open-data, 8192 context length English text embedding model that outperforms both OpenAI Ada-002 and OpenAI text-embedding-3-small on the short-context MTEB benchmark and the long context LoCo benchmark. We release the training code and model weights under an Apache 2.0 license. In contrast with other open-source models, we release the full curated training data and code that allows for full replication of nomic-embed-text-v1. You can find code and data to replicate the model at https://github.com/nomic-ai/contrastors.

Subfields: Artificial Intelligence, General Health Professions, Language and Linguistics

View Paper: https://arxiv.org/abs/2402.01613

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Health Professions

In Language and Linguistics

Citations in the corpus

mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024, methodology): “In the English community, it has been proven that training long-context encoders from scratch is promising for text retrieval (Günther et al., 2023; Nussbaum et al., 2024).”
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (2024, positive): “Short-Context Retrieval On BEIR, both variants of ModernBERT outperform existing encoders in both the DPR and ColBERT settings, including the recent GTE-en-MLM and NomicBERT models designed to serve as better backbones for retrieval (Zhang et al., 2024; Nussbaum et al., 2024).”
NOMAD Projection (2025, unknown): “We obtain high dimensional vectors for ArXiv using Nomic Embed [18] and high dimensional vectors for ImageNet using OpenClip [10].”
CNS-Obsidian: A Neurosurgical Vision-Language Model Built From Scientific Publications. (2025, unknown): “…domain knowledge into high-quality training data for vision-language models: 1 Depending on the task, models were prompted to create a user-assistant conversation, a multiple choice question with discussion, or a patient one liner with a differential diagnosis based on the image, caption, and…”
Tracking the perspectives of interacting language models (2024, methodology): “We use all-MiniLM-L6-v2 , a sentence embedding function from (Reimers and Gurevych, 2019) based on (Wang et al., 2020b) hosted on the HuggingFace Hub (Wolf et al., 2020), as the surrogate embedding function and the implementation of CMDS from Graspologic (Chung et al., 2019).”
Nomic Embed Vision: Expanding the Latent Space (2024, methodology): “5 respectively (Nussbaum et al., 2024), and the vision encoder as EVA02-ViT B/16 (Fang et al., 2023b).”
CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking (2024, unknown): “Our data filtering, negative mining, and model finetuning are implemented using the contrastors package (Nussbaum et al., 2024).”
NEAR: Neighborhood-Aware Residuals for Memory Retrieval (2026, unknown): “We evaluate nine frozen embedding models (Zhang et al., 2025; Vera et al., 2025; Zhao et al., 2026a; Nussbaum et al., 2025; Moreira et al., 2024; Babakhin et al., 2025).”
Workerville: Towards an Organizational Behavior Account of Agent Safety (2026, unknown): “5 (Nussbaum et al., 2024), 512-dimensional embeddings, cosine similarity, and FINCH clustering (Sarfraz et al., 2019).”
When Old Facts Return: Re-Reads, Reverts, and the Limits of Temporal Memory (2026, unknown): “Embeddings use nomic-embed-text with 768 dimensions (Nussbaum et al., 2024).”
Can Prosodic Style Be Inferred from Text Alone? Evidence from Unsupervised Acoustic Clusters (2026, unknown): “Sentence transformers [36] map text to dense vectors whose distances reflect semantic similarity, with variants trained on natural language inference [37], paraphrase pairs, and general-purpose contrastive objectives [38]–[40].”
MemStrata: 95% and 90.91% Source-Aware Accuracy on LongMemEval-500 and LoCoMo-1540 with a Local Qwen 3.8 27B Q4_K_M Reader (2026, unknown): “Nomic’s work provides background on the embedding family (Nussbaum et al., 2024); the experiment’s exact identity is given in Appendix E.”
MRVQ: One Resident Index for Dimension- and Rate-Elastic Vector Search (2026, unknown): “The four embedding families are MPNet [Reimers and Gurevych, 2019] ( D = 768 ), Mxbai [Lee et al., 2024] ( D = 1024 ), Nomic [Nussbaum et al., 2024] ( D = 768 ), and BGE [Xiao et al., 2023] ( D = 768 ).”
The TellTail of Embeddings: Fingerprinting Retrievers in Black-Box Systems (2026, unknown): “5 , two closely related model releases with shared lineage [44]; their similarity therefore provides a plausible explanation for the collision.”
Data and Knowledge Dual-Driven Text Embedding Approaches: A Survey (2026, unknown): “Nomic-Embed [66] represents a milestone in open-source embedding research with fully reproducible weights, code, and an 8192-token context window.”
Towards Retrieval Augmented Generation in High-Energy and Astroparticle Physics (2026, unknown): “We will also compare the performance of Specter2 with two general purpose embedders Qwen3 [19] and Nomic [20].”
A Matryoshka Hierarchical RAG for Efficient Multi-Hop Question Answering (2026, unknown): “5 [20] as the embedding model.”
CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking (2026, unknown): “The encoder uses a ModernBERT embedding checkpoint, the decoder uses Qwen 3 0.6B and the sampler is an approximately 162M parameter MLP (Ho et al., 2020; Nussbaum et al., 2025; Yang et al., 2025).”