Multilingual training for Software Engineering

By Toufique Ahmed, Prem Devanbu. 2021

97 citations recorded by the source corpus.

Well-trained machine-learning models, which leverage large amounts of open-source software data, have now become an interesting approach to automating many software engineering tasks. Several SE tasks have all been subject to this approach, with performance gradually improving over the past several years with better models and training methods. More, and more diverse, clean, labeled data is better for training; but constructing good-quality datasets is time-consuming and challenging. Ways of augmenting the volume and diversity of clean, labeled data generally have wide applicability. For some languages (e.g., Ruby) labeled data is less abundant; in others (e.g., JavaScript) the available data maybe more focused on some application domains, and thus less diverse. As a way around such data bottlenecks, we present evidence suggesting that human-written code in different languages (which performs the same function), is rather similar, and particularly preserving of identifier naming patterns; we further present evidence suggesting that identifiers are a very important element of training data for software engineering tasks. We leverage this rather fortuitous phenomenon to find evidence that available multilingual training data (across different languages) can be used to amplify performance. We study this for 3 different tasks: code summarization, code retrieval, and function naming. We note that this data-augmenting approach is broadly compatible with different tasks, languages, and machine-learning models.

Subfields: Artificial Intelligence, Information Systems, Software, Natural Language Processing, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Software

In Natural Language Processing

In Code Generation & Program Synthesis

Citations in the corpus

MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation (2023, methodology): “Code generation models have been applied to a variety of tasks, including test generation [18], docstring generation [19], code search [16], [20], type inference [21], [22], [23], and more [24].”
SantaCoder: don't reach for the stars! (2023, neutral): “Evaluating other tasks Code generation models have also been used to solve a variety of tasks (Tufano et al., 2020; Feng et al., 2020b; Ahmed & Devanbu, 2022; Hellendoorn et al., 2018; Pradel et al., 2020).”
Learning to Predict User-Defined Types (2023, background): “This pre-training approach processes large swaths of raw source code available on GitHub [20], [21], [22], [23], [24] to learn representations (neural encodings) which capture common usage patterns.”
A Scalable and Extensible Approach to Benchmarking NL2Code for 18 Programming Languages (2022, background): “…have been applied to a variety of tasks, including test generation (Tufano et al. 2020), doc-string generation (Lu et al. 2021), code search (Feng et al. 2020; Ahmed and Devanbu 2022), type inference (Wei et al. 2020; Hellendoorn et al. 2018; Pradel et al. 2020), and more (Drori et al. 2022).”
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation (2022, methodology): “Code generation models have been applied to a variety of tasks, including test generation [19], docstring generation [20], code search [17, 21], type inference [22, 23, 24], and more [25].”
Few-shot training LLMs for project-specific code-summarization (2022, background): “The majority of the foundation models [1, 2, 7, 9, 23] are evaluated on this dataset for the code summarization task.”