DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars

By Yuchi Tian, Kexin Pei, Suman Sekhar Jana, Baishakhi Ray. 2017

1,513 citations recorded by the source corpus.

Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner-case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement, and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explore different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, background): “But ML models are also vulnerable to natural OoD examples [21, 2, 48, 3].”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Even though there is promising projects in this regard such as DeepTest [42], the design-simulation-test-redesigncertification procedure is still not established by the industry nor the rule-makers.”
Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems (2017, background): “Unfortunately, similar to traditional software, such random testing approaches are not effective at finding erroneous corner-case behaviors [37], [46].”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, background): “nario and report the wrong information to people [218].”
Provable repair of deep neural networks (2021, positive): “More recent research tackles testing of DNNs [23, 42, 43, 45, 47, 54, 56, 58].”
DeepMutation: Mutation Testing of Deep Learning Systems (2018, background): “3Although, the mutation score metric defined in this paper mainly focuses on classification problems, the similar idea can be easily extended to handle numerical predication problem as well, with a user-defined threshold as the error allowance margin [32].”
Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks (2018, background): “Given their increased use in safety-critical and security applications such as autonomous vehicles [4, 74, 76], intrusion detection [28, 29], malicious string detection [65], and facial recognition [39, 70], it is important to ensure that such algorithms are robust to malicious adversaries.”
Autonomous Cars: Research Results, Issues, and Future Challenges (2019, background): “[60] proposed a DNN-based framework to test”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “Hence, the problem of black-box testing is universally relevant for all autonomous driving models and has been discussed intensively over the years [121], [122], [127], [136], [137].”
Challenges in Designing Datasets and Validation for Autonomous Driving (2019, methodology): “, 2017) and DeepTest (Tian et al., 2018) focused on the AD.”
Towards Robust CNN-based Object Detection through Augmentation with Synthetic Rain Variations (2019, methodology): “Instead of using standard noise distributions as proposed in [12]–[14], [16] or simply one specific rain filter as in [17], we use variations of realistic synthetic rain to modify the datasets and evaluate the robustness.”
Algorithms for Verifying Deep Neural Networks (2019, methodology): “• neural network testing methods that generate better test cases [16], [30], [36], [37];”