A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability

By Xiaowei Huang, Daniel Kroening, Wenjie Ruan, James Sharp, Youcheng Sun, Emese Thamo, Min Wu, Xinping Yi. Computer Science Review, 2018

607 citations recorded by the source corpus.

In the past few years, significant progress has been made on deep neural networks (DNNs) in achieving human-level performance on several long-standing tasks. With the broader deployment of DNNs on various applications, the concerns over their safety and trustworthiness have been raised in public, especially after the widely reported fatal incidents involving self-driving cars. Research to address these concerns is particularly active, with a significant number of papers released in the past few years. This survey paper conducts a review of the current research effort into making DNNs safe and trustworthy, by focusing on four aspects: verification, testing, adversarial attack and defence, and interpretability. In total, we survey 202 papers, most of which were published after 2017.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6160d130a5f0ad08f111d08153972cef3d0a6199

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziwei Liu
2. Kaiyang Zhou
3. Jingkang Yang
4. Bo Li
5. Wayne Zhang
6. Haoqi Wang
7. Yixuan Li
8. Litong Feng

In Computer Vision and Pattern Recognition

1. Ziwei Liu
2. Kaiyang Zhou
3. Chen Change Loy
4. Luc Van Gool
5. Bo Li
6. Jingkang Yang
7. Wayne Zhang
8. Hongsheng Li

Citations in the corpus

• A Safety Framework for Critical Systems Utilising Deep Neural Networks (2020, background): “The readers are referred to a recent survey [23] for the introduction and summarisation of the techniques.”
• Deep Repulsive Prototypes for Adversarial Robustness (2021, background): “2017) to formal guarantees that no adversarial examples can be found within some bounds (Huang et al. 2020) – robustness to adversarial examples remains an open problem.”
• Code Prediction by Feeding Trees to Transformers (2020, background): “Interpretability studies [36]–[39] that provide human-understandable justifications, have thus become an important property in building safe and trustworthy AI systems.”
• Generalized Out-of-Distribution Detection: A Survey (2021, positive): “Recently, one fundamental cause of the overconfidence issue on OOD data has been revealed that using mismatched BatchNorm statistics—that are estimated on ID data yet blindly applied to the OOD data in testing—can trigger abnormally high unit activations and model output accordingly (Sun et al.,…”
• Semantically Coherent Out-of-Distribution Detection (2021, background): “Those defects seriously reduce the model trustworthiness and hinder their deployment in real, especially high-risk applications [5, 6].”
• Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, background): “Deployment Improving explanations with ontologies [18] XAI and privacy [35, 58] XAI and security [2, 64, 62, 30] XAI and safety [65, 66, 2] Human-machine teaming [40, 30, 28, 66, 1, 50, 59, 24], [45, 46]* Explainable agency [67] Machine-to-machine explanation [29, 68, 1] XAI and reinforcement learni”
• Trustworthy AI: From Principles to Practices (2021, background): “Development Functional Testing [164, 235, 381] Performance Benchmarking [40, 56, 88, 93, 123, 127, 153, 238, 248, 307, 327]”