Concolic Testing for Deep Neural Networks
By Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Z. Kwiatkowska, Daniel Kroening. 2018
351 citations recorded by the source corpus.
Concolic testing combines program execution and symbolic analysis to explore the execution paths of a software program. This paper presents the first concolic testing approach for Deep Neural Networks (DNNs). More specifically, we formalise coverage criteria for DNNs that have been studied in the literature, and then develop a coherent method for performing concolic testing to increase test coverage. Our experimental results show the effectiveness of the concolic testing approach in both achieving high coverage and finding adversarial examples.
Subfields: Artificial Intelligence, Software
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Xiaowei Huang
- Marta Z. Kwiatkowska
- Wenjie Ruan
- Daniel Kroening
- Youcheng Sun
- Min Wu
- Yang Liu
- James Sharp
In Software
- Yang Liu
- Jie M. Zhang
- Lei Ma
- Daniel Kroening
- Jianjun Zhao
- Mark Harman
- Marta Z. Kwiatkowska
- James Sharp
Citations in the corpus
- Provable repair of deep neural networks (2021, neutral): “More recent research tackles testing of DNNs [23, 42, 43, 45, 47, 54, 56, 58].”
- Model Agnostic Defence Against Backdoor Attacks in Machine Learning (2019, background): “DeepConcolic [20] performs concolic testing to discover violations of robustness.”
- Global Robustness Evaluation of Deep Neural Networks with Provable Guarantees for L0 Norm (2018, methodology): “We compare our tool L0 - TRE with other state-of-the-art test case generation methods, including DeepConcolic 6 [28] and DeepXplore [20].”
- Software Engineering for AI-Based Systems: A Survey (2021, background): “A total of 36 papers addressed testing methods for AI systems, following different techniques such as combinatorial testing [104, 126, 130], concolic testing [189, 191, 192], fuzzing (e.”
- DeepHunter: a coverage-guided fuzz testing framework for deep neural networks (2019, methodology): “DeepConcolic [43] is another recently proposed white-box DL testing technique based on concolic testing.”
- Machine Learning Testing: Survey, Landscapes and Horizons (2019, methodology): “Their work pioneered to propose coverage criteria for DNN. Enlightened by this paper, a number of machine learning testing techniques have emerged, such as DeepTest [76], DeepGauge [92], DeepConcolic [111], and DeepRoad [79].”
- A Safety Framework for Critical Systems Utilising Deep Neural Networks (2020, methodology): “Formal verification [24,27,66,21,37,64,48,65,49,30] and coverage-guided testing [61,44,59,38,60,62] currently form the two major classes of V&V techniques for DL, from which a collection of evidence may be obtained that supports the partial prior knowledge.”