Audio Adversarial Examples: Targeted Attacks on Speech-to-Text

By Nicholas Carlini, David A. Wagner. 2018

1,193 citations recorded by the source corpus.

We construct targeted audio adversarial examples on automatic speech recognition. Given any audio waveform, we can produce another that is over 99.9% similar, but transcribes as any phrase we choose (recognizing up to 50 characters per second of audio). We apply our white-box iterative optimization-based attack to Mozilla's implementation DeepSpeech end-to-end, and show it has a 100% success rate. The feasibility of this attack introduce a new domain to study adversarial examples.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, background): “In addition to image classification, attacks to other DNN-powered tasks have also been actively investigated, such as visual question answering [43, 1], image captioning [10], semantic segmentation [31, 1] and etc [12, 8, 39].”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “, 2017) and audio (Carlini and Wagner, 2018) can easily be distorted with modifications that are imperceptible to humans.”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “Follow up work [84] demonstrated that it is also possible to fool speech to text systems with small perturbations to a clean audio source, where the adversarial audio is 99% similar to the original.”
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases (2019, background): “Evaluating models on out-of-domain examples built by applying minor perturbations to existing examples has also been the subject of recent study (Szegedy et al., 2014; Belinkov and Bisk, 2018; Carlini and Wagner, 2018; Glockner et al., 2018).”
Disentangling Adversarial Robustness and Generalization (2018, background): “Finally, adversarial examples have been applied to a wide variety of tasks, also beyond computer vision, e.g., [26, 18, 93, 47, 35, 54, 3, 16].”
PRADA: Protecting Against DNN Model Stealing Attacks (2018, methodology): “As examples for other domains, the decibel metric (dB) can be used on audio input [7] and the L(1) norm on malware input [18].”
The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization (2020, background): “However, they can also be extremely vulnerable to adversarial, human-imperceptible data modiﬁcations [Szegedy et al., 2013, Carlini and Wagner, 2018, Kos et al., 2018].”
Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment (2019, background): “While existing works on adversarial examples have obtained success in the image and speech domains (Szegedy et al. 2013; Carlini and Wagner 2018), it is still challenging to deal with text data due to its discrete nature.”
Rademacher Complexity for Adversarially Robust Generalization (2018, background): “This phenomenon was first discovered by Szegedy et al. (2013) in image classification problems, and similar phenomena have been observed in other areas (Carlini & Wagner, 2018; Kos et al., 2018).”
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance (2019, background): “…and Biggio et al. (2013), meth-ods to produce adversarial examples for various machine learning tasks were developed, including computer vision, speech recognition and malware detection (Goodfellow et al.; Carlini & Wagner, 2018; Grosse et al., 2016), as well as applications (Osadchy et al., 2017).”
AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learning (2018, methodology): “We borrow from white-box attacks on visual classiﬁers [15, 45], sequential models [16, 65] and object detectors [23].”
Adversarial examples for models of code (2019, background): “Generation of adversarial examples was demonstrated for image classification [19, 38] as well as for other domains [14].”
Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding (2018, methodology): “51 This algorithm has already been successfully used for adversarial examples in other settings [13, 22].”
CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (2018, background): “Recently, the authors posted the achievement that can construct targeted audio adversarial examples on DeepSpeech, an end-to-end open source ASR platform [20].”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, methodology): “This paper’s focus is evasion attacks which have been proposed against supervised ML algorithms used for image classification [13, 18, 24, 45, 97, 114], object detection [26, 83, 127], image segmentation [6, 41], speech recognition [19, 133] as well as other tasks [27, 48, 65, 129]; generative model”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Finally, a few studies have explored adversarial examples for speech recognition [51, 76] and other speech processing tasks [190]; see [127] for an overview.”
BERT: A Review of Applications in Natural Language Processing and Understanding (2021, background): “Currently, some success has been achieved in creating a methodology for constructing adversarial examples in the field of image recognition [75] and audio processing [7].”