DolphinAttack: Inaudible Voice Commands

By Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, Wenyuan Xu. 2017

771 citations recorded by the source corpus.

Speech recognition (SR) systems such as Siri or Google Now have become an increasingly popular human-computer interaction method, and have turned various systems into voice controllable systems (VCS). Prior work on attacking VCS shows that the hidden voice commands that are incomprehensible to people can control the systems. Hidden voice commands, though "hidden", are nonetheless audible. In this work, we design a totally inaudible attack, DolphinAttack, that modulates voice commands on ultrasonic carriers (e.g., f > 20 kHz) to achieve inaudibility. By leveraging the nonlinearity of the microphone circuits, the modulated low-frequency audio commands can be successfully demodulated, recovered, and more importantly interpreted by the speech recognition systems. We validated DolphinAttack on popular speech recognition systems, including Siri, Google Now, Samsung S Voice, Huawei HiVoice, Cortana and Alexa. By injecting a sequence of inaudible voice commands, we show a few proof-of-concept attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile. We propose hardware and software defense solutions, and suggest to re-design voice controllable systems to be resilient to inaudible voice command attacks.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Countering Adversarial Images using Input Transformations (2018, background): “While our study focuses exclusively on image classification, we expect similar defenses to be useful in other domains for which successful attacks have been developed, such as semantic segmentation and speech recognition (Cisse et al., 2017a; Zhang et al., 2017).”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “Attackers can generate adversarial commands against automatic speech recognition models and voice controllable system [12], [13], such as Apple Siri [14], Amazon Alexa [15], and Microsoft Cortana [16].”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
Constructing Unrestricted Adversarial Examples with Generative Models (2018, background): “This has raised concerns for safety-critical AI applications because, for example, attackers could use them to mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”
Adversarial Attacks and Defences: A Survey (2018, background): “Carlini et al [54] and Zhang et al [28] independently brought forward the vulnerabilities of automatic speech recognition and voice controllable systems.”
ReMASC: Realistic Replay Attack Corpus for Voice Controlled Systems (2019, background): “In contrast, for usability considerations, VCSs typically use less strict speaker veriﬁcation, e.g., in [32], the authors report that similar voices can activate Siri.”
Generative Adversarial Examples (2018, background): “This has raised concern for safety-critical AI applications because attackers could use them to, for example, mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”
Adversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition (2018, background): “ate on non-visual input was studied too. For instance, researchers showed that speech-recognition systems can be misled to interpret sounds that are incomprehensible to humans as actual commands [6], [66]. Other work found that electrocardiogrambased authentication can be bypassed via signal injecti”
CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (2018, background): “The recent work DolphinAttack [39] proposed a completely inaudible voice attack by modulating commands on ultrasound carriers and leveraging the nonlinearity of the microphones.”
BlackIoT: IoT Botnet of High Wattage Devices Can Disrupt the Power Grid (2018, background): “In an interesting recent work [64], Zhang et al.”
Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding (2018, methodology): “Almost all previous works on attacks against ASR systems were not DNN-based and therefore use other techniques [9], [63], [54].”
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition (2019, background): “, not neural network based) speech recognition systems (Carlini et al., 2016; Zhang et al., 2017; Song & Mittal, 2017).”
Trojaning Attack on Neural Networks (2018, background): “[62] successfully create attack commands for speech recognition system through noises or supersonic voices.”
A survey on adversarial attacks and defences (2021, background): “Carlini et al. [15] and Zhang et al. [16] independently brought forward the vulnerabilities of ASR and VCS. Attacks on autonomous vehicles have been demonstrated by Kurakin et al. [17] where the adversary manipulated traffic signs to confuse the learning model.”
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text (2018, background): “Hidden voice commands [9, 40, 42] are targeted attacks, but require synthesizing new audio (analogous to the observation that neural networks can make high confidence predictions for unrecognizable images [33]).”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, background): “Also, for those voice control supported vehicles, the attackers could successfully control the vehicle by using voices that humans cannot hear [196].”
SoK: Security Evaluation of Home-Based IoT Deployments (2019, background): “Flaws in firmware allow attackers to steal WiFi credentials [31], turn smart thermostats into spy gadgets [32], ransom them [33], run arbitrary commands on smart TVs [34], and control home assist devices covertly [35].”