Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight

By Yen-Chen Lin, Ming-Yu Liu, Min Sun, Jia-Bin Huang. 2017

55 citations recorded by the source corpus.

Deep reinforcement learning has shown promising results in learning control policies for complex sequential decision-making tasks. However, these neural network-based policies are known to be vulnerable to adversarial examples. This vulnerability poses a potentially serious threat to safety-critical systems such as autonomous vehicles. In this paper, we propose a defense mechanism to defend reinforcement learning agents from adversarial attacks by leveraging an action-conditioned frame prediction module. Our core idea is that the adversarial examples targeting at a neural network-based policy are not effective for the frame prediction model. By comparing the action distribution produced by a policy from processing the current observed frame to the action distribution produced by the same policy from processing the predicted frame from the action-conditioned frame prediction module, we can detect the presence of adversarial examples. Beyond detecting the presence of adversarial examples, our method allows the agent to continue performing the task using the predicted frame when the agent is under attack. We evaluate the performance of our algorithm using five games in Atari 2600. Our results demonstrate that the proposed defense mechanism achieves favorable performance against baseline algorithms in detecting adversarial examples and in earning rewards when the agents are under attack.

Subfields: Artificial Intelligence, Hardware and Architecture, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “Work in progress shows that there are methods to detect adversarial instances (Rawat et al., 2017) and to mitigate the attacks (Lin et al., 2017).”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “[116] leveraged multiple previous images to predict future input and detect adversarial examples, in the task of reinforcement learning.”
Deflecting Adversarial Attacks with Pixel Deflection (2018, background): “oduced by dropping a pixel is mostly high-frequency, the same cannot be said about the adversarial perturbations. Several attempts have been made to quantify distribution of adversarial perturbations [15,25] but recent work by Carlini and Wagner [5] has shown that most techniques fail to detect adve”