Retrieval on source code: a neural code search

By Saksham Sachdev, Hongyu Li, Sifei Luan, Seohyun Kim, Koushik Sen, Satish Chandra. 2018

200 citations recorded by the source corpus.

Searching over large code corpora can be a powerful productivity tool for both beginner and experienced developers because it helps them quickly find examples of code related to their intent. Code search becomes even more attractive if developers could express their intent in natural language, similar to the interaction that Stack Overflow supports. In this paper, we investigate the use of natural language processing and information retrieval techniques to carry out natural language search directly over source code, i.e. without having a curated Q&A forum such as Stack Overflow at hand. Our experiments using a benchmark suite derived from Stack Overflow and GitHub repositories show promising results. We find that while a basic word–embedding based search procedure works acceptably, better results can be obtained by adding a layer of supervision, as well as by a customized ranking strategy.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Code Generation & Program Synthesis, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Code Generation & Program Synthesis

In Program Analysis & Verification

Citations in the corpus

TypeWriter: neural type prediction with search-based validation (2019, background): “Beyond type prediction, others have proposed learning-based techniques to find programming errors [25, 28], predict variable and method names [1, 29, 35], suggest how to improve names [19], search code [10, 31], detect clones [38, 42], classify code [24, 41], predict code edits [33, 40, 43], predict”
When deep learning met code search (2019, methodology): “In NCS [29], a specific technique named after the general concept of Neural Code Search, the embedding functions Ec and Eq are implemented using a combination of token-level embeddings using fastText [7], which is similar toword2vec [27], and conventional information retrieval techniques, such as TF”
CoDesc: A Large Code–Description Parallel Dataset (2021, methodology): “We use Mean Reciprocal Rank (MRR) – the commonly used evaluation metric for code search (Husain et al., 2019; Sachdev et al., 2018; Cambronero et al., 2019) as the evaluation criteria for code search.”
Scaffle: bug localization on millions of files (2020, methodology): “Scaffle allows this search engine to be generic – in our evaluation (Section 6), we used a search tool based on vector space embedding, similar to [20].”
CoSQA: 20,000+ Web Queries for Code Search and Question Answering (2021, methodology): “…2019; Rah-man, 2019). deep learning based models encode query and code into vectors and utilize vector similarities as the metric to retrieve code (Sachdev et al., 2018; Ye et al., 2016; Gu et al., 2018; Cambronero et al., 2019; Yao et al., 2019; Liu et al., 2019a; Feng et al., 2020; Zhao and…”
Semantic code search via equational reasoning (2020, methodology): “Approaches include running a neural network over important names [68], the program AST [88], and, for Github Semantic Code Search [21], lexer output.”
Adversarial examples for models of code (2019, background): “Neural models of code have recently achieved state-of-theart performance on various tasks such as prediction of variable names and types [1, 5, 10, 32], code summarization [2, 3, 18], code generation [4, 11, 28], code search [13, 25, 34], and bug finding [29, 33, 35].”
NL2Type: Inferring JavaScript Function Types from Natural Language Information (2019, background): “To the best of our knowledge, NL2Type is the ﬁrst to predict types from comments. d) Natural Language Information and Code: Code search allows developers to ﬁnd code snippets through nat-ural language queries [43]–[46].”
Aroma: code recommendation via structural code search (2018, unknown): “Many efforts have been made to improve keyword-based code search [Bajracharya et al. 2006; Chan et al. 2012; Martie et al. 2015; McMillan et al. 2012; Sachdev et al. 2018].”
HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022, background): “In recent years, many deep learning based approaches to software engineering tasks, such as code retrieval [1, 2], code clone detection [3–6], code comment generation [7–9], software defect prediction [10–12], and code summarization [13–16], have emerged to improve software development and maintenan”