Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks

By Vahid Behzadan, Arslan Munir. 2017

311 citations recorded by the source corpus.

Deep learning classifiers are known to be inherently vulnerable to manipulation by intentionally perturbed inputs, named adversarial examples. In this work, we establish that reinforcement learning techniques based on Deep Q-Networks (DQNs) are also vulnerable to adversarial input perturbations, and verify the transferability of adversarial examples across different DQN models. Furthermore, we present a novel class of attacks based on this vulnerability that enable policy manipulation and induction in the learning process of DQNs. We propose an attack mechanism that exploits the transferability of adversarial examples to implement policy induction attacks on DQNs, and demonstrate its efficacy and impact through experimental study of a game-learning scenario.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Certified Defenses for Data Poisoning Attacks (2017, background): “However, recent adversarial attacks on reinforcement learners (Huang et al., 2017; Behzadan and Munir, 2017; Lin et al., 2017) do blend train and test vulnerabilities.”
Stochastic Activation Pruning for Robust Adversarial Defense (2018, background): “Previously, (Behzadan & Munir, 2017; Huang et al., 2017; Kos & Song, 2017) have shown that the reinforcement learning agents can also be easily manipulated by adversarial examples.”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
The Space of Transferable Adversarial Examples (2017, background): “Adversarial examples can be used to craft human-recognizable images that are misclassified by computer vision models [23, 8, 12, 16], software containing malware but classified as benign [13, 26, 9, 10], and game environments that force reinforcement learning agents to misbehave [11, 4, 15].”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “Research on other applications of neural networks has also encountered adversarial examples on different tasks beyond image classification, including deep policies in reinforcement learning and generative models [15, 14, 12, 2].”
Adversarial Attacks on Neural Network Policies (2017, background): “A recent technical report studied the scenario of an adversary interfering with the training of an agent, with the intent of preventing the agent from learning anything meaningful [3].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “For example, [219] showed successful transferability of attacks across different DQN models.”
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight (2017, background): “For DNN-based policies, the vulnerability to adversarial attack [1, 14, 22] can potentially pose a signiﬁcant danger to safety-critical applications including self-driving cars and human-robot collaboration platforms.”
Deep Reinforcement Learning for Cyber Security (2019, background): “signals in RL to trick the agent into taking sub-optimal actions, resulting in the agent being compromised [176].”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “Specifically, 1) In general, deep learning (including deep reinforcement learning) is vulnerable to adversarial examples [104], [105].”
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text (2018, background): “While neural networks are being used in increasingly many new domains, existing work on adversarial examples has focused largely on the space of images, be it image classification [41], generative models on images [26], image segmentation [1], face detection [37], or reinforcement learning by manipu”
Adversarial Attacks and Defenses in Deep Learning (2020, background): “[55] proposes perturbing the input states s t in the Q -function Q ð s t þ 1 ; a ; h t Þ , such that the learning process will produce an adversarial action a 0 .”
Adversarial Examples on Object Recognition (2020, background): “Other tasks such as reinforcement learning [10, 80, 106], speech recognition [23, 27], facial recognition [150], semantic segmentation [178] or video processing [103, 164, 170] are also explored.”
Regulatory Markets for AI Safety (2019, background): “Adversarial attacks can also interfere with the policies learned by deep reinforcement learning agents (Behzadan & Munir, 2017; Huang et al., 2017).”
Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning (2020, background): “Transferability of adversarial examples across different DRL models were studied in (Behzadan and Munir 2017a).”
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations (2020, background): “These early approaches achieved much worse results than ours: for Pong, Behzadan & Munir [5] can improve reward under attack from −21 (lowest) to −5, yet is still far away from the optimal reward (+21).”
Evaluation and Design of Robust Neural Network Defenses (2018, background): “Existing work on adversarial examples has focused largely on the space of images, be it image classification [125], generative models on images [64], image segmentation [2], face detection [115], or reinforcement learning by manipulating the images the RL agent sees [8, 49].”
Transparency in Deep Reinforcement Learning Networks (2018, methodology): “Fast Gradient Sign Method generates an adversarial input x∗ by maximizing a network loss function J(θ, x, y) with the one step update shown in Eq.[2].”