Hidden Voice Commands

By Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David A. Wagner, Wenchao Zhou. 2016

682 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Towards the Science of Security and Privacy in Machine Learning (2016, background): “As a natural extension to [67] (see above), it was shown that rubbish audio can be classiﬁed with conﬁdence by a speech recognition system [68].”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “, virtual assistants [5], malware detectors [16] or ad-blockers [39]).”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “he striking discovery that test images could be perturbed in a visually-imperceptible way and yet fool state-of-the-art neural network image classiers (Szegedy et al., 2014; Goodfellow et al., 2015; Carlini et al., 2016; Kurakin et al., 2016; Papernot et al., 2016a, 2017; Moosavi-Dezfooli et al., 20”
Spectral Signatures in Backdoor Attacks (2018, background): “We also point out that backdoor poisoning is related to adversarial examples [13, 26, 21, 12, 29, 4, 24, 32].”
Certified Defenses for Data Poisoning Attacks (2017, background): “…striking recent security vulnerability discovered in machine learning systems is adversarial test images that can fool image classiﬁers despite being imperceptible from normal images (Szegedy et al., 2014; Goodfellow et al., 2015; Carlini et al., 2016; Kurakin et al., 2016; Papernot et al., 2016a).”
Adversarial examples in the physical world (2016, background): “For example, Carlini et al. (2016) demonstrate an attack that can create audio inputs that mobile phones recognize as containing intelligible voice commands, but that humans hear as an unintelligible voice.”
Attacking Graph-based Classification via Manipulating the Graph Structure (2019, background): “ing studies on adversarial machine learning mainly focused on machine learning for non-graph data. In particular, studies have demonstrated that machine learning is vulnerable to adversarial examples [7,12,27,30,37,40,43,46,51,53,56, 68], poisoning attacks [8,18,31,38,67,70], privacy attacks for use”
Stealing Hyperparameters in Machine Learning (2018, background): “Evasion attacks: In these attacks [33], [22], [3], [51], [52], [17], [26], [23], [37], [56], [9], [44], [28], [36], an attacker aims to inject carefully crafted noise into a testing instance (e.g., an email spam, a social spam, a malware, or a face image) such that the classiﬁer predicts a…”
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks (2017, background): “demonstrated that lowering the sampling rate helps to defend against the adversarial voice commands [5].”
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition (2016, background): “Speech Recognition Carlini et al. recently demonstrated physically realizable attacks on speech-recognition systems [5].”
Certified Defenses against Adversarial Examples (2018, methodology): “Some attacks (Carlini & Wagner, 2016; Tramèr et al., 2017) evade these defenses and witness a large f ( A opt ( x )) .”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “Attackers can generate adversarial commands against automatic speech recognition models and voice controllable system [12], [13], such as Apple Siri [14], Amazon Alexa [15], and Microsoft Cortana [16].”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “For example, to defend against “hidden voice commands” [83] suggests that designers could modify a device to notify the user any time a voice command is received.”
SoK: Security and Privacy in Machine Learning (2018, background): “ns 8 need to introduce adapted, often larger, perturbations. As a natural extension to [67] (see above), it was shown that rubbish audio can be classiﬁed with conﬁdence by a speech recognition system [68]. Consequences are not as important in terms of security then [37]: the audio does not correspon”
On Evaluating Adversarial Robustness (2019, background): “…(or spam) classifier to identify a malicious file (or spam email) as benign (Dahl et al., 2013), cause an ad-blocker to incorrectly identify an advertisement as natural content (Tramèr et al., 2018), or cause a digital assistant to incorrectly recognize commands it is given (Carlini et al., 2016).”
Constructing Unrestricted Adversarial Examples with Generative Models (2018, background): “This has raised concerns for safety-critical AI applications because, for example, attackers could use them to mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”