Robustness

By Stephan Morgenthaler. 2020

That the conclusion based on a data analysis be robust and stable is not merely a desirable feature, it is essential. To merit this quality label, a conclusion must be supported by strong data‐based evidence and not simply be a discovery gleaned from a preconceived model and weakly supported by a part of the data. Robustness in statistics refers to the definition and investigation of procedures that lead to such stability. This article gives a brief overview of the concepts and procedures that are relevant in judging robustness. These have mostly been developed over the last five decades. WIREs Comp Stat 2011 3 85–94 DOI: 10.1002/wics.144

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Meta-learning of Sequential Strategies (2019, background): “Going beyond expected losses means that we have to change our certainty-equivalent —that is, the way we aggregate uncertain losses into a single value [Hansen and Sargent, 2008].”
Does language help generalization in vision models? (2021, methodology): “The different model variants (AR-L2, AR-LI4, AR-LI8) correspond to distinct adversarial attacks (refer to (Engstrom et al., 2019b) for more details).”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, background): “…also drove the need for certiﬁed defenses (Wong et al., 2018; Mirman et al., 2018; Gowal et al., 2018; Zhang et al., 2019b; Cohen et al., 2019; Salman et al., 2019), but the guarantees that these techniques provide do not yet match the empirical robustness obtained through adversarial…”
Sensitivity analysis using approximate moment condition models (2018, methodology): “Under conditions in Imbens and Angrist (1994) and Heckman and Vytlacil (2005), instrumental variables estimates with different instruments are consistent for average treatment effects for different subpopulations.”
An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference? (2020, background): “And practitioners may well benefit from robustifying their analysis (Mosteller and Tukey, 1977; Hansen and Sargent, 2008; Chen et al., 2011) even if they pass our check.”
Targeted perturbations reveal brain-like local coding axes in robustified, but not standard, ANN-based brain models (2025, unknown): “In addition, to investigate whether increasing robustness improves the prediction accuracy of the encoding models, we also used publicly available models that were robustified through adversarial training (Engstrom et al., 2019; Ilyas et al., 2019).”