NL2Type: Inferring JavaScript Function Types from Natural Language Information

By Rabee Sohail Malik, Jibesh Patra, Michael Pradel. 2019

138 citations recorded by the source corpus.

JavaScript is dynamically typed and hence lacks the type safety of statically typed languages, leading to suboptimal IDE support, difficult to understand APIs, and unexpected runtime behavior. Several gradual type systems have been proposed, e.g., Flow and TypeScript, but they rely on developers to annotate code with types. This paper presents NL2Type, a learning-based approach for predicting likely type signatures of JavaScript functions. The key idea is to exploit natural language information in source code, such as comments, function names, and parameter names, a rich source of knowledge that is typically ignored by type inference algorithms. We formulate the problem of predicting types as a classification problem and train a recurrent, LSTM-based neural model that, after learning from an annotated code base, predicts function types for unannotated code. We evaluate the approach with a corpus of 162,673 JavaScript files from real-world projects. NL2Type predicts types with a precision of 84.1% and a recall of 78.9% when considering only the top-most suggestion, and with a precision of 95.5% and a recall of 89.6% when considering the top-5 suggestions. The approach outperforms both JSNice, a state-of-the-art approach that analyzes implementations of functions instead of natural language information, and DeepTyper, a recent type prediction approach that is also based on deep learning. Beyond predicting types, NL2Type serves as a consistency checker for existing type annotations. We show that it discovers 39 inconsistencies that deserve developer attention (from a manual analysis of 50 warnings), most of which are due to incorrect type annotations.

Subfields: Information Systems, Code Generation & Program Synthesis, Program Analysis & Verification, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Code Generation & Program Synthesis

In Program Analysis & Verification

In Signal Processing

Citations in the corpus

Do Machine Learning Models Produce TypeScript Types that Type Check? (2023, methodology): “NL2Type [32] is another system for predicting JavaScript types that improves on DeepTyper.”
TypeWriter: neural type prediction with search-based validation (2019, methodology): “sful type predictions (GitHub: PrefectHQ/ct, awslabs/sockeye). truth. We compute precision, recall, and F1-score, weighted by the number of type occurrences in the dataset. Similarly to previous work [21], if the prediction model cannot predict a type for a type slot (i.e., returns “unknown”), we re”
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (2024, background): “Finally, some works have additionally studied other tasks and scenarios like type prediction (Mir et al., 2022; Wei et al., 2023a; Malik et al., 2019), code summarization (LeClair et al., 2019; Iyer et al., 2016; Barone and Sennrich, 2017; Hasan et al., 2021; Alon et al., 2018), code security…”
LambdaNet: Probabilistic Type Inference using Graph Neural Networks (2020, methodology): “iction space. Hellendoorn et al. (2018) and Jangda & Anand (2019) model programs as sequences and AST trees and apply deep learning models (RRNs and Tree-RNNs) for TypeScript and Python programs. Malik et al. (2019) make use of a different source of information and take documentation strings as ”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, neutral): “and stemming for type prediction [60].”
OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints (2020, methodology): “For example, names of variables provide information about their types [Xu et al. 2016], natural language in method-level comments provide information about function types [Malik et al. 2019], and lexically nearby tokens provide information about a variable’s type [Hellendoorn et al. 2018].”
Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python (2021, methodology): “In previous work [31, 51], the type prediction task is formulated as a classification problem.”
Multi-task Learning based Pre-trained Language Model for Code Completion (2020, methodology): “pes to make helpful suggestions for completing partial code. For example, when accessing the field of an object in a Java IDE, code completion suggests suitable field names based on the object’s type [35]. Taking the code completion example of a Java IDE ( ItelliJ IDEA) in Figure 1 as an example, th”
Learning to Predict User-Defined Types (2023, background): “Unknown : In previous works with ﬁxed type vocabularies [9], [10], [11], [12], [13] type inference models would predict UNK if the type exceeded its classiﬁcation capabilities.”
Solver-based gradual type migration (2021, methodology): “It is also possible to produce type migrations using supervised machine learning [Hellendoorn et al. 2018; Malik et al. 2019; Pradel et al. 2020; Wei et al. 2020].”
Typilus: neural type hints (2020, background): “ese two ideas in a Graph Neural Network. Similar to Raychev et al. [51], DeepTyper cannot predict rare or previously unseen types and its implementation does not subtokenise identifiers. Malik et al. [46] learn to predict types for JavaScript code justfromthedocumentationcomments.Theyfindthat(good) ”
Learning type annotation: is big data enough? (2021, methodology): “LambdaNet [25] (like other recent works) used sophisticated inductive biases [3, 4, 10] to achieve state-of-the-art (SOTA) type inference, improving substantially upon earlier approaches like Hellendoorn [11] and Malik et al. [17].”
SimTyper: sound type inference for Ruby using type equality prediction (2021, background): “Our results show that type equality prediction can help type inference systems effectively produce more usable types.”