CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking

By Tarun Suresh, Revanth Gangi Reddy, Yifei Xu, Zach Nussbaum, Andriy Mulyar, Brandon Duderstadt, Heng Ji. International Conference on Learning Representations, 2024

44 citations recorded by the source corpus.

Effective code retrieval plays a crucial role in advancing code generation, bug fixing, and software maintenance, particularly as software systems increase in complexity. While current code embedding models have demonstrated promise in retrieving code snippets for small-scale, well-defined tasks, they often underperform in more demanding real-world applications such as bug localization within GitHub repositories. We hypothesize that a key issue is their reliance on noisy and inconsistent datasets for training, which impedes their ability to generalize to more complex retrieval scenarios. To address these limitations, we introduce CoRNStack, a large-scale, high-quality contrastive training dataset for code that spans multiple programming languages. This dataset is curated using consistency filtering to eliminate noisy positives and is further enriched with mined hard negatives, thereby facilitating more effective learning. We demonstrate that contrastive training of embedding models using CoRNStack leads to state-of-the-art performance across a variety of code retrieval tasks. Furthermore, the dataset can be leveraged for training code reranking models, a largely underexplored area compared to text reranking. Our finetuned code reranking model significantly improves the ranking quality over the retrieved results. Finally, by employing our code retriever and reranker together, we demonstrate significant improvements in function localization for GitHub issues, an important component of real-world software development.

Subfields: Information Retrieval & Search, Representation & Self-Supervised Learning, Code Generation & Program Synthesis

View Paper: https://arxiv.org/abs/2412.01007

Citations in the corpus

• Lost in Translation, Found in Summary: Summary-Driven Supervision for Cross-Language Code Retrieval (2026, unknown): “Unlike prior uses of natural language, which pair code with docstrings [49], Summary IR serves as a pivot for grouping functionally equivalent code snippets across languages.”
• RepoNav: From Snippet Retrieval to File-Centered Repository Navigation for Code Agents (2026, unknown): “Following Chen et al. (2025), each function is embedded as a single chunk using CodeRankEmbed (Suresh et al., 2025).”
• ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search (2026, unknown): “C ODE [44] Open 7.0B 768 32K C ODE R ANK E MBED [44] Open 137M 768 8K CODEBERT - BASE [11] Open 125M 768 512 G RAPH C ODE BERT-BASE [17] Open 125M 768 512 C ODE T5-SMALL [51] Open 60M 512 512 4) Code Preprocessing: All source code is preprocessed before embedding to reduce implementation noise…”
• HyperFL: Query-Adaptive Representation Learning for Software Fault Localization (2026, unknown): “Hy-perFL consistently outperforms lexical retrieval, pretrained embedding models, and the state-of-the-art issue localization method SweRank across all evaluation metrics.”
• DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search (2026, unknown): “Adding CoRN-Stack (Suresh et al., 2025) during pre-training may further improve code retrieval, and we leave this for future work.”
• Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents (2026, unknown): “5b [11], pplx-embed-v1-4b [5], and nomic-embed-code [16].”
• Finding the Right Tables and Columns: A Benchmark and Corpus-Adaptive Embeddings for SQL Schema Retrieval (2026, unknown): “…pre-trained text embedders (Izacard et al., 2021; Wang et al., 2024; Merrick et al., 2024; Xiao et al., 2024) dominate first-stage retrieval, with late-interaction (Khattab and Zaharia, 2020) and code-specialized (Liu et al., 2025; Suresh et al., 2025) variants developed for structured inputs.”
• Are We Measuring Strategy or Phrasing? The Gap Between Surface- and Approach-Level Diversity in LLM Math Reasoning (2026, unknown): “We compare two general-purpose models, BAAI/bge-m3 (Chen et al., 2024) and the OpenAI embedding API (OpenAI, 2024), with two code-oriented models, nomic-ai/nomic-embed-code (Suresh et al., 2025) and jina-embeddings-v2-base-code (Günther et al., 2024).”
• Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrieval (2026, unknown): “This yielded 17 models (125M to 30B parameters), selected on four criteria: (i) architectural paradigm : bi-encoders (e.g., CodeRankEmbed [30]), encoder-decoders (e.g., CodeT5 [33]), and decoder-only LLMs in embedding mode (e.g., Code Llama [28], Qwen3 [35]); (ii) model scale : favouring the base…”
• SHERLOC: Structured Diagnostic Localization for Code Repair Agents (2026, unknown): “We therefore complement file-level metrics with structure-agnostic chunk-level metrics over line ranges (coverage recall, precision, and average tightness) that represent predictions and ground truth as (start, end) line-number spans, making no assumption about which syntactic unit contains the…”
• FMI_SU_Yotkova_Kastreva at SemEval-2026 Task 13: Lightweight Detection of LLM-Generated Code via Stylometric Signals (2026, unknown): “), and CodeR-ankLLM (Suresh et al., 2025) with standard classifiers.”
• Identifier-Free Code Embedding Models for Scalable Search (2026, unknown): “0 [20], SFR-Embedding-Mistral [21], jina-embeddings-v2-base-code [22], and nomic-embed-code [23].”
• Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval (2026, unknown): “Data-centric efforts range from dataset curation (CoRNStack [33]) and synthesis (GME [43]) to advanced hard negative mining, including in-batch methods and LLM-judge-based approaches (UniME-V2 [8]).”
• CLARC: C/C++ Benchmark for Robust Code Search (2026, unknown): “It utilized the CoRNStack dataset (Suresh et al., 2025) and a curriculum-based hard negative mining strategy, which progressively introduces more challenging negative examples to the model over time using softmax-based sampling during training.”
• SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution (2026, unknown): “SWE-Adept achieves stronger performance across all the evaluated programming languages, demonstrating its broad applicability.”