Formal Security Analysis of Neural Networks using Symbolic Intervals

By Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, Suman Sekhar Jana. 2018

532 citations recorded by the source corpus.

Due to the increasing deployment of Deep Neural Networks (DNNs) in real-world security-critical domains including autonomous vehicles and collision avoidance systems, formally checking security properties of DNNs, especially under different attacker capabilities, is becoming crucial. Most existing security testing techniques for DNNs try to find adversarial examples without providing any formal security guarantees about the non-existence of such adversarial examples. Recently, several projects have used different types of Satisfiability Modulo Theory (SMT) solvers to formally check security properties of DNNs. However, all of these approaches are limited by the high overhead caused by the solver. In this paper, we present a new direction for formally checking security properties of DNNs without using SMT solvers. Instead, we leverage interval arithmetic to compute rigorous bounds on the DNN outputs. Our approach, unlike existing solver-based approaches, is easily parallelizable. We further present symbolic interval analysis along with several other optimizations to minimize overestimations of output bounds. We design, implement, and evaluate our approach as part of ReluVal, a system for formally checking security properties of Relu-based DNNs. Our extensive empirical results show that ReluVal outperforms Reluplex, a state-of-the-art solver-based system, by 200 times on average. On a single 8-core machine without GPUs, within 4 hours, ReluVal is able to verify a security property that Reluplex deemed inconclusive due to timeout after running for more than 5 days. Our experiments demonstrate that symbolic interval analysis is a promising new direction towards rigorously analyzing different security properties of DNNs.

Subfields: Artificial Intelligence, Software, Safety, Robustness & Interpretability, Program Analysis & Verification, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Cho-Jui Hsieh
  3. Huan Zhang
  4. Pushmeet Kohli
  5. J. Kolter
  6. Mohit Bansal
  7. Suman Sekhar Jana
  8. Sven Gowal

In Software

  1. Suman Sekhar Jana
  2. Huan Zhang
  3. Cho-Jui Hsieh
  4. Gagandeep Singh
  5. Markus Püschel
  6. Shiqi Wang
  7. Baishakhi Ray
  8. Pushmeet Kohli

In Safety, Robustness & Interpretability

  1. Huan Zhang
  2. Cho-Jui Hsieh
  3. Suman Sekhar Jana
  4. Shiqi Wang
  5. Pushmeet Kohli
  6. Sven Gowal
  7. Timon Gehr
  8. Clark W. Barrett

In Program Analysis & Verification

  1. Huan Zhang
  2. Suman Sekhar Jana
  3. Shiqi Wang
  4. Cho-Jui Hsieh
  5. Gagandeep Singh
  6. Markus Püschel
  7. Pushmeet Kohli
  8. Timon Gehr

In Signal Processing

  1. Suman Sekhar Jana
  2. Sven Gowal
  3. Shiqi Wang
  4. Kexin Pei
  5. Timothy A. Mann
  6. Junfeng Yang
  7. Justin Whitehouse
  8. Clark W. Barrett

Citations in the corpus