Sponge Examples: Energy-Latency Attacks on Neural Networks

By Ilia Shumailov, Yiren Zhao, Daniel Bates, Nicolas Papernot, Robert D. Mullins, Ross Anderson. 2020

192 citations recorded by the source corpus.

The high energy costs of neural network training and inference led to the use of acceleration hardware such as GPUs and TPUs. While such devices enable us to train large-scale neural networks in datacenters and deploy them on edge devices, their designers' focus so far is on average-case performance. In this work, we introduce a novel threat vector against neural networks whose energy consumption or decision latency are critical. We show how adversaries can exploit carefully-crafted sponge examples, which are inputs designed to maximise energy consumption and latency, to drive machine learning (ML) systems towards their worst-case performance. Sponge examples are, to our knowledge, the first denial-of-service attack against the ML components of such systems. We mount two variants of our sponge attack on a wide range of state-of-the-art neural network models, and find that language models are surprisingly vulnerable. Sponge examples frequently increase both latency and energy consumption of these models by a factor of 30×. Extensive experiments show that our new attack is effective across different hardware platforms (CPU, GPU and an ASIC simulator) on a wide range of different language tasks. On vision tasks, we show that sponge examples can be produced and a latency degradation observed, but the effect is less pronounced. To demonstrate the effectiveness of sponge examples in the real world, we mount an attack against Microsoft Azure's translator and show an increase of response time from 1ms to 6s (6000×). We conclude by proposing a defense strategy: shifting the analysis of energy consumption in hardware from an average-case to a worst-case perspective.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

The Curse of Recursion: Training on Generated Data Makes Models Forget (2023, methodology): “The repeating problem has been observed in nearly all text generation models [Keskar et al., 2019, Shumailov et al., 2021] and to rule this out as the cause of Model Collapse , we further provide numerical experiments when models are explicitly encouraged to produce non-repeating sequences with…”
Manipulating SGD with Data Ordering Attacks (2021, background): “first attacked the availability of computer vision and natural language processing models at inference time with sponge examples [30].”
Dataset Inference: Ownership Resolution in Machine Learning (2021, background): “Model extraction can also be a reconnaissance step used to prepare for further attacks, e.g., ﬁnding adversarial examples (Papernot et al., 2017; Shumailov et al., 2020).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “This strategy aims to draw the community’s attention to their potential impact on autonomous vehicles, as well as scenarios requiring making decisions in timely manner.”
LLM Censorship: A Machine Learning Challenge or a Computer Security Problem? (2023, background): “However, such methods can also be unreliable [9, 46, 10].”