Large Language Model guided Protocol Fuzzing
By Ruijie Meng, Martin Mirchev, Marcel Böhme, Abhik Roychoudhury. Network and Distributed System Security Symposium, 2024
315 citations recorded by the source corpus.
—How to find security flaws in a protocol implementation without a machine-readable specification of the protocol? Facing the internet, protocol implementations are particularly security-critical software systems where inputs must adhere to a specific structure and order that is often informally specified in hundreds of pages in natural language (RFC). Without some machine-readable version of that protocol, it is difficult to automatically generate valid test inputs for its implementation that follow the required structure and order. It is possible to partially alleviate this challenge using mutational fuzzing on a set of recorded message sequences as seed inputs. However, the set of available seeds is often quite limited and will hardly cover the great diversity of protocol states and input structures. In this paper, we explore the opportunities of systematic interaction with pre-trained large language models (LLMs), which have ingested millions of pages of human-readable protocol specifications, to draw out machine-readable information about the protocol that can be used during protocol fuzzing. We use the knowledge of the LLMs about protocol message types for well-known protocols. We also checked the LLM’s capability in detecting “states” for stateful protocol implementations by generating sequences of messages and predicting response codes. Based on these observations, we have developed an LLM-guided protocol implementation fuzzing engine. Our protocol fuzzer C HAT AFL constructs grammars for each message type in a protocol, and then mutates messages or predicts the next messages in a message sequence via interactions with LLMs. Experiments on a wide range of real-world protocols from P RO F UZZ B ENCH show significant efficacy in state and code coverage. Our LLM-guided stateful fuzzer was compared with state-of-the-art fuzzers AFLN ET and NSF UZZ . C HAT AFL covers 47.60% and 42.69% more state transitions, 29.55% and 25.75% more
Subfields: Electrical and Electronic Engineering, Hardware and Architecture, Software
View Paper: https://www.semanticscholar.org/paper/4fcd49afa8d0a960c6d07aa4e7a37956f5307a8a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Hardware and Architecture
In Software
Citations in the corpus
- Agentless: Demystifying LLM-based Software Engineering Agents (2024, methodology): “Researchers have applied LLMs to perform fuzzing in domains such as DL libraries [35, 36], OS Kernel [100, 70], compilers [93, 99, 75], network protocols [66], and mobile applications [59].”
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “CHATAFL [181] is an LLM-guided protocol fuzzer that constructs grammars for message types and mutates messages or predicts the next messages based on LLM interactions, achieving better state and code coverage compared to state-of-the-art fuzzers (e.g., AFLNET [208], NSFUZZ [213]).”