Houdini: Fooling Deep Structured Prediction Models

By Moustapha Cissé, Yossi Adi, Natalia Neverova, Joseph Keshet. 2017

280 citations recorded by the source corpus.

Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Mitigating adversarial effects through randomization (2017, background): “, 2017) and semantic segmentation (Xie et al., 2017; Fischer et al., 2017; Cisse et al., 2017).”
Countering Adversarial Images using Input Transformations (2018, background): “Unfortunately, recent research has demonstrated that existing models are not robust to small, adversarially designed perturbations of the input (Biggio et al., 2013; Szegedy et al., 2014; Goodfellow et al., 2015; Kurakin et al., 2016a; Cisse et al., 2017a).”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Besides successful generation of adversarial images for classification, Houdini has also been shown to successfully attack a popular deep Automatic Speech Recognition system [151].”
ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism (2017, background): “Concomitantly, several studies have demonstrated the lack of robustness of deep neural networks to adversarial examples [42, 16, 7] and raised questions about their tendency to exhibit (undesirable) biases [37].”
Constructing Unrestricted Adversarial Examples with Generative Models (2018, background): “This has raised concerns for safety-critical AI applications because, for example, attackers could use them to mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”
Towards Fast Computation of Certified Robustness for ReLU Networks (2018, background): “…found adversarial examples in many machine learning tasks applied to different areas, including object detection (Xie et al., 2017), image captioning (Chen et al., 2018a), speech recognition (Cisse et al., 2017), malware detection (Wang et al., 2017) and reading comprehension (Jia & Liang, 2017).”
ReMASC: Realistic Replay Attack Corpus for Voice Controlled Systems (2019, background): “Despite their convenience, VCSs also raise new security concerns due to their vulnerability to multiple types of spoofing attacks, such as replay attacks, self-triggered attacks [1], hidden voice commands [2, 3], and acoustic adversarial examples [4, 5, 6].”
Adversarial Attacks on Node Embeddings via Graph Poisoning (2018, unknown): “& Wagner, 2017; Liang et al., 2018; Ciss´e et al., 2017; Lin et al., 2017; Chen et al., 2017a).”
Simple Black-box Adversarial Attacks (2019, background): “Computer vision models for image segmentation and object detection have also been shown to be vulnerable against adversarial perturbations (Cisse et al., 2017a; Xie et al., 2017).”
Improving Transferability of Adversarial Examples With Input Diversity (2018, background): “Besides image classification, adversarial examples also exist in object detection [38], semantic segmentation [38, 5], speech recognition [5], deep reinforcement learning [19], etc.”
Adversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition (2018, background): “use excessive makeup [31] or attempt to blind the camera using light-emitting eyeglasses [79]. The susceptibility to attacks of learning systems that operate on non-visual input has also been studied [14, 16, 17, 81]. For instance, researchers showed that speech-recognition systems can be misled to ”
Generative Adversarial Examples (2018, background): “This has raised concern for safety-critical AI applications because attackers could use them to, for example, mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”
Generative Adversarial Perturbations (2017, methodology): “[11] proposes a method for directly optimizing performance measures, even when they are combinatorial and non-decomposable.”
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks (2019, background): “, 2016), Parseval networks (Cisse et al., 2017), and adversarial training (Madry et al.”
Robust Physical-World Attacks on Deep Learning Visual Classification (2018, methodology): “Attacking detectors are out of the scope of this paper, though recent work has generated digital adversarial examples against detection/segmentation algorithms [6,20,38], and our recent work has extended RP 2 to attack the YOLO detector [7].”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, methodology): “This paper’s focus is evasion attacks which have been proposed against supervised ML algorithms used for image classification [13, 18, 24, 45, 97, 114], object detection [26, 83, 127], image segmentation [6, 41], speech recognition [19, 133] as well as other tasks [27, 48, 65, 129]; generative model”
Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving (2019, methodology): “To approach this problem, we formulate the attack task as an optimization problem, which has been shown to be effective in previous machine learning security studies [21, 24, 26, 50, 51, 53].”