Does Distributionally Robust Supervised Learning Give Robust Classifiers?

By Weihua Hu, Gang Niu, Issei Sato, Masashi Sugiyama. 2016

253 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, background): “bounded divergences between source and target distributions) (Mansour et al., 2009; Hu et al., 2016).”
Fairness Without Demographics in Repeated Loss Minimization (2018, background): “This is closely related to recent observations that the DRO bound can be loose for classification losses such as the zeroone loss due to the worst-case distribution consisting purely of misclassified examples (Hu et al., 2018).”
Learning Robust Representations by Projecting Superficial Statistics Out (2018, background): “Other papers bound worst-case performance under bounded shifts as measured by divergence measures on the train v.s. test distributions (Ben-David et al., 2010a; Mansour et al., 2009; Hu et al., 2016).”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “• Group DRO (Hu et al., 2018; Sagawa et al., 2020) uses distributionally robust optimization (DRO) to explicitly minimize the loss on the worst-case domain, RDRO(θ) := max d∈Dtrain ÊPd [`(θ; (x, y))] .”
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (2019, methodology): “In contrast, most DRO work has focused on the classic (underparameterized) model setting (Namkoong & Duchi, 2017; Hu et al., 2018; Duchi et al., 2019).”
Out-of-Distribution Generalization via Risk Extrapolation (REx) (2020, background): “When only a single training domain is available (singlesource domain generalization), it is common to assume that P (Y |X) is fixed, and let F be all distributions within some f -divergence ball of the training P (X) (Hu et al., 2016; Bagnell, 2005).”
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, methodology): “Because these notions of robustness can be pessimistic [23], others investigate group DRO (GDRO), which optimizes worst-case performance over a known set of subgroups [23, 43].”
DORO: Distributional and Outlier Robust Optimization (2021, unknown): “Outliers have a greater effect on the performance of DRO than ERM (Hu et al., 2018), due to its focus on the tail performance, so removing this negative impact of outliers is crucial for the success of DRO in its real-world applications.”
Robustness to Spurious Correlations via Human Annotations (2020, background): “(3) Prior work has shown that such shifts over groups can be used to capture a wide range of test time distributions including label shifts (Hu et al., 2018), topics within a corpus (Oren et al.”
MEMO: Test Time Robustness via Adaptation and Augmentation (2021, background): “, 2013; Gulrajani & Lopez-Paz, 2021), and distributionally robust optimization (Ben-Tal et al., 2013; Hu et al., 2018; Sagawa et al., 2020), to name just a few.”
Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift (2020, methodology): “The ﬁrst assumption is that the training data are provided in groups, which, as discussed above, mirrors analogous assumptions made in group DRO [25], federated learning [43], and meta-learning [66], among other settings.”
Distributionally Robust Language Modeling (2019, methodology): “Instead, we propose to consider meaningful subpopulations, deﬁned by topics in a corpus (Hu et al., 2018).”
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual (2019, background): “Our objective is more related to distributionally robust optimization (Duchi and Namkoong, 2018; Hu et al., 2018), which does not assume access to target data and optimizes the worst-case performance under unknown , bounded distribution shift.”
Gradient Matching for Domain Generalization (2021, unknown): “…(Peng et al., 2019), and offers comparison to a variety of SOTA domain generalization methods, including IRM (Arjovsky et al., 2019), Group DRO (Hu et al., 2018; Sagawa et al., 2019), Mixup (Yan et al., 2020), MLDG (Li et al., 2018a), Coral (Sun and Saenko, 2016), MMD (Li et al., 2018b), DANN…”