Code Representation Learning At Scale

By Dejiao Zhang, Wasi Uddin Ahmad, Ming Tan, Hantian Ding, Ramesh Nallapati, Dan Roth, Xiaofei Ma, Bing Xiang. 2024

Subfields: Representation & Self-Supervised Learning, Code Generation & Program Synthesis, Software Engineering

Citations in the corpus

• CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking (2024, unknown): “Further, our 137M parameter encoder outperforms the 1.3B CodeSage-Large model, which is ten times larger.”
• SweRank: Software Issue Localization with Code Ranking (2025, unknown): “Wang et al. (2023c) and Zhang et al. (2024) learn improved code representations by incorporating a mix of training objectives, such as span denoising, text-code matching and causal LM pretraining, over large-scale code corpora such as CodeSearchNet (Husain et al., 2019) and The Stack (Kocetkov et al”
• Improving Code Localization with Repository Memory (2025, unknown): “Orthogonally, our approach leverages commit histories to construct a repository-specific memory grounded in the codebase’s evolution.”
• CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2025, unknown): “…20 retrieval models, which covers six paradigms: (1) unsupervised retrievers [14]); (2) supervised retrievers [15, 16]; (3) code-specific retrievers [17, 18]; (4) LLM-based retrievers [15, 19, 20]; (5) instruction-following retrievers [19, 21, 22]; and (6) proprietary API-based models [23, 24].”