Differentiable Abstract Interpretation for Provably Robust Neural Networks

By Matthew Mirman, Timon Gehr, Martin T. Vechev. 2018

582 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

In Program Analysis & Verification

Citations in the corpus

Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, methodology): “loss (α, 0). ℓ′(Fθ,(x˜,y))= log(exp(max y′,y дθ(x˜) y′−дθ(x˜))+1) (14) Interval Bound Propagation-Based Verification (IBP-Based Verify) [16]: Gowal et al. [16] share a similar design as Mirman et al. [34]: they express the constraint Bϵ as a bounded interval domain (one specified domain considered b”
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations (2019, background): “…are able to tie veriﬁcation into training non-trivial deep networks by backpropagating through certiﬁ-cates, which are generated with duality-based bounds (Wong & Kolter, 2018; Wong et al., 2018), abstract interpretations (Mirman et al., 2018), and interval bound propagation (Gowal et al., 2018).”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “…have been proposed in the literature (Wong & Kolter, 2018; Wang et al., 2018a;b; Raghunathan et al., 2018a;b; Wong et al., 2018; Dvijotham et al., 2018b;a; Croce et al., 2018; Gehr et al., 2018; Mirman et al., 2018; Singh et al., 2018; Gowal et al., 2018; Weng et al., 2018a; Zhang et al., 2018).”
Towards neural networks that provably know when they don't know (2019, positive): “Empirically, adversarial training (Madry et al., 2018) performs well and provably robust models have been developed (Hein & Andriushchenko, 2017; Wong & Kolter, 2018; Raghunathan et al., 2018; Mirman et al., 2018; Cohen et al., 2019).”
Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem (2018, methodology): “Apart from methods which provide robustness guarantees for neural networks [14, 37, 31, 26] which give still only reasonable guarantees for small networks, up to our knowledge the only approach which has not been broken again [6, 5, 2] is adversarial training [25] using robust optimization technique”
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models (2018, methodology): “[20]) are not necessary to reach tight verified bounds.”
Fast is better than free: Revisiting adversarial training (2020, background): “While certiﬁed defenses have been scaled to reasonably sized networks (Wong et al., 2018; Mirman et al., 2018; Gowal et al., 2018; Cohen et al., 2019; Salman et al., 2019), the guarantees don’t match the empirical robustness obtained through adversarial training.”
Towards Stable and Efficient Training of Verifiably Robust Neural Networks (2019, methodology): “…Kolter, 2018; Mirman et al., 2018; Wang et al., 2018a; Dvijotham et al., 2018b; Weng et al., 2018; Zhang et al., 2018), interval bound propagation (Mirman et al., 2018; Gowal et al., 2018), ReLU stability regularization (Xiao et al., 2019c), and distributionally ∗Work partially done during an…”
MixTrain: Scalable Training of Formally Robust Neural Networks (2018, methodology): “0348 DIFFAI [18] Out of memory∗ Wong et al.”
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification (2021, background): “Additionally, another line of works train networks to enhance verified accuracy, typically using cheap incomplete verifiers at training time [42, 39, 25, 43, 18, 25, 47, 4, 32].”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, background): “The difficulty of adversarial evaluation also drove the need for certified defenses (Wong et al., 2018; Mirman et al., 2018; Gowal et al., 2018; Zhang et al., 2019b; Cohen et al., 2019; Salman et al., 2019), but the guarantees that these techniques provide do not yet match the empirical robustness o”
Relating Adversarially Robust Generalization to Flat Minima (2021, background): “Besides certified and provable defenses [22, 138, 66, 147, 146, 130, 41, 39, 84, 113, 72, 23], adversarial training (AT) has become the de-facto standard, as discussed in the main paper.”
An abstract domain for certifying neural networks (2019, methodology): “, can be computed in parallel), we can directly plug in these transformers into the latest systems which train neural networks using abstract interpretation on GPUs [Mirman et al. 2018]. As our transformers are substantially more precise than those of Mirman et al. [2018], we expect they can help im”
Adversarial Attacks and Defenses in Deep Learning (2020, unknown): “[107–109]”
Lipschitz-Certifiable Training with a Tight Outer Bound (2020, background): “In contrast, the layer-wise bound propagation approach calculates the upper bounds on the worst case error through relaxation on the layer-wise operations and forward propagation for the perturbation set that can be made of (cid:96) ∞ - or (cid:96) 2 -balls [13, 44, 36, 25, 32, 37].”