Adversarial attacks on medical machine learning

By Samuel G. Finlayson, John Bowers, Joichi Ito, Jonathan L. Zittrain, Andrew Beam, Isaac S. Kohane. Science, 2019

963 citations recorded by the source corpus.

Emerging vulnerabilities demand new conversations With public and academic attention increasingly focused on the new role of machine learning in the health information economy, an unusual and no-longer-esoteric category of vulnerabilities in machine-learning systems could prove important. These vulnerabilities allow a small, carefully designed change in how inputs are presented to a system to completely alter its output, causing it to confidently arrive at manifestly wrong conclusions. These advanced techniques to subvert otherwise-reliable machine-learning systems—so-called adversarial attacks—have, to date, been of interest primarily to computer science researchers (1). However, the landscape of often-competing interests within health care, and billions of dollars at stake in systems' outputs, implies considerable problems. We outline motivations that various players in the health care system may have to use adversarial attacks and begin a discussion of what to do about them. Far from discouraging continued innovation with medical machine learning, we call for active engagement of medical, technical, legal, and ethical experts in pursuit of efficient, broadly available, and effective health care that machine learning will enable.

Subfields: Artificial Intelligence, Health Informatics, Public Health, Environmental and Occupational Health

View Paper: https://www.semanticscholar.org/paper/e718828e8f776d9f80daa3f8e0af6895f5d34c44

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Public Health, Environmental and Occupational Health

Citations in the corpus

Clean-Label Backdoor Attacks on Video Recognition Models (2020, background): “This has raised concerns for the development of DNNs in applications such as face recognition [21, 3], autonomous driving [5], video analysis [11], and medical diagnosis [7, 17].”
Key challenges for delivering clinical impact with artificial intelligence (2019, background): “For example, in one study, images of benign moles were misdiagnosed as malignant by adding adversarial noise or even just rotation [83].”
Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems (2019, background): “cs that are quite different from natural images, for example, unique biological textures. A recent work has conﬁrmed that medical deep learning systems can also be compromised by adversarial attacks (Finlayson et al., 2019). As shown in Figure1, across three medical image datasets Fundoscopy (Kaggle”
Ethical and legal challenges of artificial intelligence-driven healthcare (2020, positive): “[110] have shown in a recent publication that the system’s output can completely be changed so that it classifies a mole as malignant with 100% confidence by making a small change in how inputs are presented to the system [109].”
AI in Health: State of the Art, Challenges, and Future Directions (2019, background): “[126] provide a more detailed discussion on the potential concerns about the “incentives for more sophisticated adversarial attacks” in healthcare.”
Characteristics of publicly available skin cancer image datasets: a systematic review. (2021, background): “10 78 This has implications for the use of watermarked atlas images, which might require processing such as cropping prior to use.”
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks (2020, background): “These attacks raise security concerns for the development of DNNs in safety-critical scenarios such as face recognition [53], autonomous driving [15,13], and medical diagnosis [17,45,38,48].”
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence (2019, background): “…viewed as something of a curiosity in the deep learning community, they have since been widely recognized as a profound challenge that may undermine the applicability of DNNs to safety–critical areas such as clinical medicine (Finlayson et al. 2019) or autonomous vehicles (Eykholt et al. 2018).”
FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, neutral): “60”
Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension (2020, neutral): “For example, changes to medical imaging that are invis-ible, or appear random, to the human eye may change the likelihood of the resultant diagnostic output entirely 47,48 .”
Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review (2020, background): “in particular, suffers from various security and privacy issues such as adversarial attacks on DL based medical systems [137], poisoning attack [138], and privacy breaches.”
Improving Adversarial Robustness Requires Revisiting Misclassified Examples (2020, background): “This vulnerability of DNNs raises serious security concerns about their practicability in security critical applications (Chen et al., 2015; Kurakin et al., 2016; Jiang et al., 2019; Finlayson et al., 2019; Ma et al., 2019).”