Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems

By Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, Feng Lu. Pattern Recognition, 2019

547 citations recorded by the source corpus.

Deep neural networks (DNNs) have become popular for medical image analysis tasks like cancer diagnosis and lesion detection. However, a recent study demonstrates that medical deep learning systems can be compromised by carefully-engineered adversarial examples/attacks with small imperceptible perturbations. This raises safety concerns about the deployment of these systems in clinical settings. In this paper, we provide a deeper understanding of adversarial examples in the context of medical images. We find that medical DNN models can be more vulnerable to adversarial attacks compared to models for natural images, according to two different viewpoints. Surprisingly, we also find that medical adversarial attacks can be easily detected, i.e., simple detectors can achieve over 98% detection AUC against state-of-the-art attacks, due to fundamental feature differences compared to normal examples. We believe these findings may be a useful basis to approach the design of more explainable and secure medical deep learning systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2172552b917ef3757b0af47d17fce18586d56cba

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review (2021, background): “A detailed discussion about the sensor fusion challenges, including adversarial attacks and possible preventions is beyond the scope of this paper (see [16,202,205-209] for a more comprehensive overview).”
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, background): “A type of well-studied adversary is called the adversarial attack (Szegedy et al., 2013; Goodfellow et al., 2014; Ma et al., 2018; Jiang et al., 2019; Wang et al., 2019b; 2020; Duan et al., 2020; Ma et al., 2020).”
Clean-Label Backdoor Attacks on Video Recognition Models (2020, background): “This has raised concerns for the development of DNNs in applications such as face recognition [21, 3], autonomous driving [5], video analysis [11], and medical diagnosis [7, 17].”
Transformers in Medical Imaging: A Survey (2022, background): “Advances in adversarial attacks have revealed the vulnerability of existing learning-based medical imaging systems against imperceptible perturbation in the input images [404]–[406].”
Improving Adversarial Robustness Requires Revisiting Misclassified Examples (2020, background): “This vulnerability of DNNs raises serious security concerns about their practicability in security critical applications (Chen et al., 2015; Kurakin et al., 2016; Jiang et al., 2019; Finlayson et al., 2019; Ma et al., 2019).”
VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization (2021, background): “A system which can perform this task in an intelligent way is hugely in demand, as its applications range from video surveillance [1] to defect segmentation [2], [3], inspection [2], quality control [4], medical imagining [5], financial transactions [6] etc.”
Secure, privacy-preserving and federated machine learning in medical imaging (2020, background): “This complicates data handling, increasing the probability of errors and constitutes a manipulation of the imaging data, which, at worst, represents an adversarial update to the algorithm 43 , reducing its performance and robustness.”
Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (2020, background): “This has raised security concerns on the deployment of DNNs in security critical scenarios, such as face recognition (Sharif et al., 2016), autonomous driving (Evtimov et al., 2018), video analysis (Jiang et al., 2019) and medical diagnosis (Ma et al., 2019).”
DCSAU-Net: A Deeper and More Compact Split-Attention U-Net for Medical Image Segmentation (2022, background): “Medical image analysis plays an essential role in terms of diagnosing and treating these diseases [1].”
Deep neural network models for computational histopathology: A survey (2019, background): “Several recent studies (Finlayson et al., 2019; Jungo and Reyes, 2019; Ma et al., 2019) have demonstrated that DL system can be compromised by carefully designed adversarial examples, i.e., even small imperceptible perturbations can deceive neural networks in predicting wrong outputs with high…”