CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

By Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, Carl A. Gunter. USENIX Security Symposium, 2018

399 citations recorded by the source corpus.

The popularity of ASR (automatic speech recognition) systems, like Google Voice, Cortana, brings in security concerns, as demonstrated by recent attacks. The impacts of such threats, however, are less clear, since they are either less stealthy (producing noise-like voice commands) or requiring the physical presence of an attack device (using ultrasound). In this paper, we demonstrate that not only are more practical and surreptitious attacks feasible but they can even be automatically constructed. Specifically, we find that the voice commands can be stealthily embedded into songs, which, when played, can effectively control the target system through ASR without being noticed. For this purpose, we developed novel techniques that address a key technical challenge: integrating the commands into a song in a way that can be effectively recognized by ASR through the air, in the presence of background noise, while not being detected by a human listener. Our research shows that this can be done automatically against real world ASR applications. We also demonstrate that such CommanderSongs can be spread through Internet (e.g., YouTube) and radio, potentially affecting millions of ASR users. We further present a new mitigation technique that controls this threat.

Subfields: Artificial Intelligence, Computer Networks and Communications, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/92e016e7e526350c04d166631ab42aa0ccba3b4f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving (2019, methodology): “input [18, 20, 29, 41, 57], we are then motivated to explore the possibility of strategically controlling the spoofed points to fool the machine learning model in LiDAR-based perception.”
Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding (2018, methodology): “We compare the amount of noise with CommanderSong [23], as their approach is also able to create 270 targeted attacks using Kaldi and therefore the same DNN-HMM-based ASR system.”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, methodology): “This paper’s focus is evasion attacks which have been proposed against supervised ML algorithms used for image classification [13, 18, 24, 45, 97, 114], object detection [26, 83, 127], image segmentation [6, 41], speech recognition [19, 133] as well as other tasks [27, 48, 65, 129]; generative model”
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition (2019, background): “Con-currently, the CommanderSong (Yuan et al., 2018) attack developed adversarial examples that are effective over-the-air but at a cost of introducing a signiﬁcant perturbation to the original audio.”