Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks

By Rüdiger Ehlers. 2017

695 citations recorded by the source corpus.

We present an approach for the verification of feed-forward neural networks in which all nodes have a piece-wise linear activation function. Such networks are often used in deep learning and have been shown to be hard to verify for modern satisfiability modulo theory (SMT) and integer linear programming (ILP) solvers. The starting point of our approach is the addition of a global linear approximation of the overall network behavior to the verification problem that helps with SMT-like reasoning over the network behavior. We present a specialized verification algorithm that employs this approximation in a search process in which it infers additional node phases for the non-linear nodes in the network from partial node phase assignments, similar to unit propagation in classical SAT solving. We also show how to infer additional conflict clauses and safe node fixtures from the results of the analysis steps performed during the search. The resulting approach is evaluated on collision avoidance and handwritten digit recognition case studies.

Subfields: Artificial Intelligence, Software, Safety, Robustness & Interpretability, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Cho-Jui Hsieh
  2. J. Kolter
  3. Percy Liang
  4. Huan Zhang
  5. Matthias Hein
  6. Pushmeet Kohli
  7. Sergey Levine
  8. Eric Wong

In Software

  1. Eric Wong
  2. J. Kolter
  3. Suman Sekhar Jana
  4. Huan Zhang
  5. Leslie Rice
  6. Cho-Jui Hsieh
  7. Shiqi Wang
  8. A. Dimakis

In Safety, Robustness & Interpretability

  1. Huan Zhang
  2. J. Kolter
  3. Eric Wong
  4. Cho-Jui Hsieh
  5. Matthias Hein
  6. Suman Sekhar Jana
  7. Percy Liang
  8. Pushmeet Kohli

In Program Analysis & Verification

  1. Eric Wong
  2. J. Kolter
  3. Huan Zhang
  4. Suman Sekhar Jana
  5. Shiqi Wang
  6. Cho-Jui Hsieh
  7. Timon Gehr
  8. Clark W. Barrett

Citations in the corpus