Robust Statistics

By Peter Filzmoser. 2018

In lieu of an abstract, here is the entry's first paragraph: Robust statistics are procedures that maintain nominal Type I error rates and statistical power in the presence of violations of the assumptions that underpin parametric inferential statistics. Since George Box coined the term in 1953, research on robust statistics has centered on the assumption of normality, although the violation of other parametric assumptions (e.g., homogeneity of variance) has their own implications for the accuracy of parametric procedures. This entry looks at the importance of robust statistics in educational and social science research and explains the robustness argument. It then describes robust descriptive statistics, their inferential extensions, and two common resampling procedures that are robust alternatives to classic parametric methods. Disciplines Statistics and Probability Comments This is an entry in: Frey, B. (2018). The SAGE encyclopedia of educational research, measurement, and evaluation (Vols. 1-4). Thousand Oaks,, CA: SAGE Publications, Inc.: https://dx.doi.org/10.4135/ 9781506326139 © 2018 by SAGE Publications, republished with permission. Content may not be distributed, resold, repurposed, used for commercial MOOCs, nor any other commercial purposes without permission. Please contact SAGE for any further usage or questions. This article is available at Fisher Digital Publications: https://fisherpub.sjfc.edu/statistics_facpub/8 1434 Robust Statistics Couch, A., & Keniston, K. (1960). Yeasayers and naysayers: Agreeing response set as a personality variable. Journal of Abnormal and Social Psychology, 60, 151-174. Gosling, S. D., Rentfrow, P. ]., & Swann, W. B., Jr. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37, 504-528. Llorente, E., Warren, C. S., de Eulate, L. P., & Gleaves, D. H. (2013 ). A Spanish version of the sociocultural attitudes towards appearance questionnaire-3 (SATAQ-3 ): Translation and psychometric evaluation. Journal of Clinical Psychology, 69(3 ), 240-251. doi: 10.1002/jclp.21944 Rodebaugh, T. L, Woods, C. M, Heimberg, R. G., Liebowitz, M. R., & Schneier, F. R. (2006). The factor structure and screening utility of the Social Interaction Anxiety Scale. Psychological Assessment, 18(2), 231-237. Tay, L., & Drasgow, F. (2012). Theoretical, statistical, and substantive issues in the assessment of construct dimensionality: Accounting for the item response process. Organizational Research Methods, 15(3), 363-384. VonSonderen, E., Sanderman, R., & Coyne, J.C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let's learn from cows in the rain. PloSONE, 8(7), e68967. doi:l0.1371/journal.pone.0068967 Weijters, B., Baumgartner, H., & Schillewaet, N. (2013). Reversed item bias: An integrative model. Psychological Method, 18(3), 320-334. doi:l0.1037/

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Robust Aggregation for Federated Learning (2019, neutral): “[42] P. J. Huber.”
Byzantine Stochastic Gradient Descent (2018, background): “Optimization and learning in the presence of adversarial noise is a well-studied problem [4, 5, 15, 20, 22, 28].”
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates (2018, methodology): “This constraint precludes direct application of standard robust learning algorithms (such as M-estimators (Huber, 2011)), which assume access to all data.”
Zeno: Byzantine-suspicious stochastic gradient descent (2018, background): “The robust estimator of the mean is a well-studied topic in statistics [8].”
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning (2018, methodology): “Robust regression has been extensively studied in statistics as a method to provide resilience against noise and outliers [26], [27], [51], [55].”
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer (2019, background): “For instance, median is a more robust statistic of data compared to mean [25], [53], [57], and therefore, f Mean in FedAvg can be replaced with weighted median aggregation f Median , which computes coordinate-wise weighted median of the updates.”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, neutral): “(2) Using the inequalities dD(μ1, μ2) ≤ dTV (μ1, μ2) [19] and dP (μ1, μ2) ≤ dTV (μ1, μ2) [25], the proof is immediate, using Theorem 1 and Lemma 1.”
Are Labels Necessary for Neural Architecture Search? (2020, background): “The straight lines are fit with robust linear regression [13] (same for Figure 3 and Figure 4).”
Robust Estimators in High Dimensions without the Computational Intractability (2016, neutral): “Keywords-unsupervised learning, statistical learning, density estimation robust algorithm I. INTRODUCTION”
Agnostic Estimation of Mean and Covariance (2016, background): “There is a large literature (see the books [Hub11, HRRS11, MMY06] for a comprehensive treat-ment) on robust statistics, with the goal of ﬁnding estimators that are stable under perturbations of the data.”
Learning from untrusted data (2016, methodology): “In this case, we can identify outliers i Ig by considering the value of fi (w̃i ) − fi (ŵi ), and Lemma 4.5 verifies that we can use this to perform outlier removal.”
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016, neutral): “For instance, mapping a 2D indoor environment with a robot equipped with wheel encoders and a laser scanner, with sufﬁcient accuracy ( 10 cm ) and sufﬁcient robustness (say, low failure rate), can be considered largely solved.”
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2019, neutral): “Note that the idea of measuring the local Lipschitz continuity is similar to the local shift sensitivity criterion in existing literature on robust statistics, which dates back to 1960’s (Hampel, 1974; Huber, 2011).”
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels (2017, background): “In the special cases, ρ ( ‘ ) degenerates to the classical robust M-estimator: Huber (Huber et al., 1964) and the log-sum penalty (Candes et al., 2008).”
Learning with Bad Training Data via Iterative Trimmed Loss Minimization (2018, background): “In the linear setting, it has a breakdown point of 1/2 asymptotically (Huber, 2011), and is consistent (Vı́šek, 2006), i.”
Finite-sample Analysis of M-estimators using Self-concordance (2018, background): “Crucially, φ(·) must be globally Lipschitz-continuous, which guarantees robustness of the M -estimator, see [23].”
Sub-Gaussian estimators of the mean of a random vector (2017, background): “One may easily show (see, e.g., Hsu [8]) that, for any δ ∈ (0 , 1) if k = ⌈ 8log(1 /δ ) , then the resulting estimator µ ( δ ) N satis-ﬁes that, with probability at least 1 − δ , µ (cid:12)(cid:12)(cid:12)(cid:12)b where σ 2 denotes the variance of X .”