Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation

By Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy (Dj) Dvijotham, Pushmeet Kohli. 2019

184 citations recorded by the source corpus.

Neural networks are part of many contemporary NLP systems, yet their empirical successes come at the price of vulnerability to adversarial attacks. Previous work has used adversarial training and data augmentation to partially mitigate such brittleness, but these are unlikely to find worst-case adversaries due to the complexity of the search space arising from discrete text perturbations. In this work, we approach the problem from the opposite direction: to formally verify a system’s robustness against a predefined class of adversarial attacks. We study text classification under synonym replacements or character flip perturbations. We propose modeling these input perturbations as a simplex and then using Interval Bound Propagation – a formal model verification method. We modify the conventional log-likelihood training objective to train models that can be efficiently verified, which would otherwise come with exponential search complexity. The resulting models show only little difference in terms of nominal accuracy, but have much improved verified accuracy under perturbations and come with an efficiently computable formal guarantee on worst case adversaries.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, neutral): “Note that this speciﬁcation addresses the output distribution of a generative model, in which it differs from prior work on speciﬁcations in NLP models which concern individual predictions of discriminative models (Huang et al., 2019; Jia et al., 2019).”
Competition-level code generation with AlphaCode (2022, methodology): “We also analysed sensitivity to synonym substitutions in the problem description and specification, using synonym pairs in Huang et al. (2019) relying on the PPDB database (Ganitkevitch et al., 2013).”
On Robust Prefix-Tuning for Text Classification (2022, methodology): “…of defense techniques are developed, including model functional improvement (Li & Sethy, 2019; Jones et al., 2020), certiﬁcation (Jia et al., 2019; Huang et al., 2019; Shi et al., 2020; Xu et al., 2020; Ye et al., 2020), adversary detection (Pruthi et al., 2019; Zhou et al., 2019), and…”
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, methodology): “Recent work (Huang et al., 2019; Jia et al., 2019) has applied IBP in the NLP domain to certify the robustness of models.”
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning (2020, methodology): “This is the evaluation method adopted in (Zhang et al., 2019; Huang et al., 2019; Jia et al., 2019; Li et al., 2020; Zang et al., 2020; Li and Qiu, 2021, inter alia.”
Certifying LLM Safety against Adversarial Prompting (2023, methodology): “They use techniques such as interval-bound propagation (Gowal et al., 2018; Huang et al., 2019; Dvijotham et al., 2018; Mirman et al., 2018), curvature bounds (Wong & Kolter, 2018; Raghunathan et al.”
Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond (2020, methodology): “We then conduct certified defense with δtrain = {1, 6} using IBP as in [19, 17] and our efficient IBP+Backward perturbation analysis.”
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020, background): “Recently, Jia et al. (2019) and Huang et al. (2019) train NLP models which are provably robust to word replacements.”