NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System

By Xi Victoria Lin, Chenglong Wang, Luke S. Zettlemoyer, Michael D. Ernst. 2018

214 citations recorded by the source corpus.

We present new data and semantic parsing methods for the problem of mapping English sentences to Bash commands (NL2Bash). Our long-term goal is to enable any user to perform operations such as file manipulation, search, and application-specific scripting by simply stating their goals in English. We take a first step in this domain, by providing a new dataset of challenging but commonly used Bash commands and expert-written English descriptions, along with baseline methods to establish performance levels on this task.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

SPoC: Search-based Pseudocode to Code (2019, background): “Previous work on executable semantic parsing mainly focuses on translating short text descriptions to a one-line program [42, 35, 43, 44, 21, 8], and while recent work explored generating longer programs from text descriptions [22, 40, 29, 15, 16, 14], these programs are mostly evaluated on…”
A Survey of Machine Learning for Big Code and Naturalness (2017, background): “ and mining link Iyer et al. [96] Source code snippets with their StackOverﬂow title. link Kushman and Barzilay [110] Dataset for generating regular expressions from natural language. link Lin et al. [119] Textto Bash commands corpus. link Ling et al. [120] Textdescriptions and code for card game. l”
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, background): “, 2020), programming instructions (Lin et al., 2018; Shao and Nakashole, 2020), inter alia.”
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search (2019, neutral): “[16], Ling et al.”
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (2018, background): “Recently, researchers have constructed some datasets for code generation including IFTTT (Quirk et al., 2015), DJANGO (Oda et al., 2015), HEARTHSTONE (Ling et al., 2016), NL2Bash (Lin et al., 2018), and CoNaLa (Yin et al., 2018).”
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow (2018, background): “ore fine-grained, sub-code-block sized candidates, aiming to recover code solutions that exactly answer the intent. Finally, some work has asked programmers to manually write NL descriptions for code [20, 30], or vice-versa [41]. This allows for the generation of high-quality data, but is time consu”
Incorporating External Knowledge through Pre-training for Natural Language to Code Generation (2020, background): “However, recently there has been a move towards applying semantic parsing to automatically generating source code in general-purpose programming languages (Yin et al., 2018; Yao et al., 2018; Lin et al., 2018; Agashe et al., 2019; Yao et al., 2019).”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, background): “, 2020) have enjoyed dramatic success on semantic parsing (Lin et al., 2018; Hwang et al., 2019; Shaw et al., 2020; Suhr et al., 2020).”
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language Instructions (2021, background): “For example, mapping natural language to database commands (Kim et al., 2020), visualizations (Shao and Nakashole, 2020), console commands (Lin et al., 2018), robot actions (Shridhar et al., 2020; Stepputtis et al., 2020), inter alia .”
SpreadsheetCoder: Formula Prediction from Semi-structured Context (2021, unknown): “…2017; Bunel et al., 2018; Bavishi et al., 2019; Shin et al., 2018; Chen et al., 2019), natural language descriptions (Gulwani & Marron, 2014; Yu et al., 2018; Yin et al., 2018; Lin et al., 2018; Liang et al., 2018; Wang et al., 2020), and images (Wu et al., 2017; Liu & Wu, 2019; Sun et al., 2018).”
Can we generate shellcodes via natural language? An empirical study (2021, background): “NMT is an emerging approach for code generation [88,56] and other programming tasks, such as code completion [23,77], the generation of UNIX commands [52,53] or commit messages [44,58,45], etc.”
Execution-based Evaluation for Data Science Code Generation Models (2022, background): “…input speciﬁcations, including natural language (Wang et al., 2015; Oda et al., 2015; Zhong et al., 2017; Yin et al., 2018; Yu et al., 2018; Lin et al., 2018), unﬁnished code (Iyer et al., Consequently, recent works turn to execution-based metrics instead, where the code would be correct…”
AgentBench: Evaluating LLMs as Agents (2023, background): “Despite attempts on translating natural language to Shell commands [45], few prior efforts evaluate on authentic executable environments.”
LEVER: Learning to Verify Language-to-Code Generation with Execution (2023, methodology): “Intermediate execution results were used to guide program search at both training (Chen et al., 2019; 2021b) and inference time (Wang et al., 2018).”
Natural Language to Code Translation with Execution (2022, methodology): “, 2018) and NL2Bash (Bash; Lin et al., 2018), covering a range of programming languages.”
An Application-Independent Approach to Building Task-Oriented Chatbots with Interactive Continual Learning (2020, methodology): “[34, 29] designed an NLI for submitting web forms and interacting with webpages and [22] proposed an NLI for Bash commands.”
Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[41] Xi Victoria Lin, Chenglong Wang, Luke Zettlemoyer, and Michael D Ernst.”
A Survey on Semantic Parsing (2018, background): “Other such datasets are introduced in Iyer et al. [2016], Yao et al. [2018], Lin et al. [2018], Oda et al. [2015].”
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021, background): “plt.scatter(age, duration) Prediction plt.scatter(duration, age) There has been work on translating natural language to code in different languages (Zettlemoyer and Collins, 2012; Wang et al., 2015; Oda et al., 2015; Yin et al., 2018; Zhong et al., 2017; Yu et al., 2018; Lin et al., 2018).”
Investigating Math Word Problems using Pretrained Multilingual Language Models (2021, background): “This task therefore can be seen as a special case of code synthesis (Desai et al., 2016; Lin et al., 2018; Shin et al., 2019; Wang et al., 2020) in a restricted domain, where the code to be synthesized is the expression tree.”