Testing Robustness Against Unforeseen Adversaries

By Daniel Kang, Yi Sun, Dan Hendrycks, Tom B. Brown, Jacob Steinhardt. 2019

144 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “Synthetic distribution shifts have been studied extensively in the literature [28, 33, 38, 48, 57, 93].”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, background): “Kang et al. (2019) construct an adversarial version of the aforementioned benchmark.”
Unsolved Problems in ML Safety (2021, background): “To study adversarial robustness more broadly [61], researchers could consider attacks that are perceptible [142] or whose speciﬁcations are not known beforehand [97, 112].”
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “, 2019) or reduced robustness to other small perturbations (Yin et al., 2019; Tramèr & Boneh, 2019; Kang et al., 2019).”
Meta-Learning in Neural Networks: A Survey (2020, background): “New benchmarks for adversarial defenses have recently been proposed [284] where defenses should generalize to unforseen attacks.”
Identifying Statistical Bias in Dataset Replication (2020, background): “…& Efros, 2011), and explored the impact of synthetic perturbations on generalization, such as adversarial examples (Kurakin et al., 2016; Tsipras et al., 2019; Ilyas et al., 2019; Su et al., 2018) or various other corruption robustness measures (Hendrycks & Dietterich, 2019; Kang et al., 2019).”
Meta-learning via Language Model In-context Tuning (2021, background): “For example, the test distribution might constitute new input compositions (Lake and Baroni, 2018), rare subgroups (Sagawa et al., 2019), other types of distribution shifts (Hendrycks and Diet-terich, 2019), or even adversarial examples (Kang et al., 2019).”
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (2020, background): “In addition to evaluating this comparison in TA and RA (evaluated at ∞ PGD attack [21]), we also measure the robustness in eventual classification against 12 unforeseen attacks that are not used in AT [17].”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “bounded by `0, `1-norm or adversarial patches 361 [11, 24, 84, 27], (2) multiple `p-norm perturbations [125, 81], (3) adversarially optimized common 362 corruptions [62, 63], (4) a broad set of perturbations unseen during training [73].”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, unknown): “(2023) have been proposed, along with numerous robustness benchmarks (Kaufmann et al., 2019; Croce et al., 2020).”