Learn&Fuzz: Machine learning for input fuzzing

By Patrice Godefroid, Hila Peleg, Rishabh Singh. International Conference on Automated Software Engineering, 2017

416 citations recorded by the source corpus.

Fuzzing consists of repeatedly testing an application with modified, or fuzzed, inputs with the goal of finding security vulnerabilities in input-parsing code. In this paper, we show how to automate the generation of an input grammar suitable for input fuzzing using sample inputs and neural-network-based statistical machine-learning techniques. We present a detailed case study with a complex input format, namely PDF, and a large complex security-critical parser for this format, namely, the PDF parser embedded in Microsoft's new Edge browser. We discuss and measure the tension between conflicting learning and fuzzing goals: learning wants to capture the structure of well-formed inputs, while fuzzing wants to break that structure in order to cover unexpected code paths and find bugs. We also present a new algorithm for this learn&fuzz challenge which uses a learnt input probability distribution to intelligently guide where to fuzz inputs.

View Paper: https://www.semanticscholar.org/paper/5bd13e6313008e1555e530dda6d84c5004aa09ed

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

REDQUEEN: Fuzzing with Input-to-State Correspondence (2019, background): “Even machine learning techniques were investigated to infer semi-valid inputs as test cases [8], [24], [27].”
NL2Type: Inferring JavaScript Function Types from Natural Language Information (2019, methodology): “Other work uses neural networks to predict parts of code [29], to detect vulnerabilities [30], to generate inputs for fuzz testing [56], [57], to detect and ﬁx syntactic programming mistakes [58], and to predict whether a ﬁle is likely to contain a bug [59]. g) JavaScript Analysis: Program analyses…”
kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels (2017, background): “Others even sought to infer the whole input grammar from program traces [13,24,38].”
FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage (2017, methodology): “Learn&Fuzz [23] uses sequencebased learning methods to learn the structure of PDF objects, Autogram [32] proposes a taint analysis-based approach to learning input grammars, while Glade [8] uses an iterative approach and repeated calls to an oracle to learn a context-free grammar for a set of inputs”
RESTler: Stateful REST API Fuzzing (2019, background): “Automatically learning input grammars from input samples is another complementary research area [25], [6], [24], [36].”
On Learning Meaningful Code Changes Via Neural Machine Translation (2019, methodology): “Due to the power of these representations, many works have applied these models to solve SE problems [39][26][37][35][51][44][58][43].”
Deep Learning to Find Bugs (2017, background): “Existing approaches address a variety of development tasks, including code completion based on probabilistic models of code [9, 34, 36], predicting fixes of compilation errors via neural networks [8, 14], and the generation of inputs for fuzz testing via probabilistic, generative language models [30”
The Art, Science, and Engineering of Fuzzing: A Survey (2018, methodology): “Neural [65] and Learn&Fuzz [94] use a neural network-based machine learning technique to learn a model from a given set of test files and then generate test cases from the inferred”
Fuzz4ALL: Universal Fuzzing with Large Language Models (2023, methodology): “Deep learning models have been used to fuzz PDF parsers [26], OpenCL [17], C [48], network protocols [85], and JavaScript [37].”
ContractFuzzer: Fuzzing Smart Contracts for Vulnerability Detection (2018, methodology): “Godefroid et al [8] designs a PDF fuzzer based on Recurrent neural network (RNN) to test if there exists any bugs on latest Edge browser of Microsoft.”
Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts (2017, background): “…machine learning to programs, targeting a variety of development tasks such as code completion [10, 29, 32], fixing syntactic errors [9, 18], generating inputs for fuzz testing of programs [17, 24, 28], and generating programs or short code snippets to assist developers or test compilers [4, 34].”
T-Fuzz: Fuzzing by Program Transformation (2018, methodology): “Learning based approaches has been shown to be effective in generating well structured inputs (e.g., XML files) for fuzzers in Skyfire [44] and Learn&Fuzz [45].”
NAUTILUS: Fishing for Deep Bugs with Grammars (2019, methodology): “[23], or by using machine learning techniques [33].”
Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models (2022, methodology): “Learn&Fuzz [23], DeepSmith [15] and DeepFuzz [47] each trains a RNN to generate programs for PDF file parsers, OpenCL and C, respectively.”
Fuzzing: A Survey for Roadmap (2022, methodology): “, chunk replacement, chunk deletion, and chunk insertion) [74, 77, 78, 181, 197].”
CollAFL: Path Sensitive Fuzzing (2018, methodology): “proposes a RNN (Recurrent Neural Network) solution [18] to generate valid seed files, and could help generate inputs to pass format checks, improving the code coverage.”
Learning to Fuzz from Symbolic Execution with Application to Smart Contracts (2019, background): “Several works [18, 23, 64] learn generative models from existing input corpus.”