An Investigation of Why Overparameterization Exacerbates Spurious Correlations

By Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, Percy Liang. 2020

450 citations recorded by the source corpus.

We study why overparameterization -- increasing model size well beyond the point of zero training error -- can hurt test error on minority groups despite improving average test error when there are spurious correlations in the data. Through simulations and experiments on two image datasets, we identify two key properties of the training data that drive this behavior: the proportions of majority versus minority groups, and the signal-to-noise ratio of the spurious correlations. We then analyze a linear setting and theoretically show how the inductive bias of models towards "memorizing" fewer examples can cause overparameterization to hurt. Our analysis leads to a counterintuitive approach of subsampling the majority group, which empirically achieves low minority error in the overparameterized regime, even though the standard approach of upweighting the minority fails. Overall, our results suggest a tension between using overparameterized models versus using all the training data for achieving low worst-group error.

Subfields: Artificial Intelligence, Scaling & Learning Theory, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Scaling & Learning Theory

In Safety, Robustness & Interpretability

Citations in the corpus

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, background): “fore, larger models might be better simply because they are better at memorization (Sagawa et al., 2020).”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “• Group DRO (Hu et al., 2018; Sagawa et al., 2020) uses distributionally robust optimization (DRO) to explicitly minimize the loss on the worst-case domain, RDRO(θ) := max d∈Dtrain ÊPd [`(θ; (x, y))] .”
Towards Out-Of-Distribution Generalization: A Survey (2021, background): “There are various synthetic data generation mechanisms in literature, and one can refer to [13, 222, 160] for more synthetic settings.”
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training (2021, unknown): “, 2021), to detect non-spurious correlation (Sagawa et al., 2020), etc.”
Datamodels: Predicting Predictions from Training Data (2022, background): “Tackling this question may necessitate a better understanding of the training dynamics and implicit biases behind overparameterized training Bartlett et al. (2021); Sagawa et al. (2020).”
Implicit Regularization via Neural Feature Alignment (2020, background): “…selection ability might explain in part the performance of neural networks on a range of supervised tasks, it may also make them brittle under spurious correlation (e.g., Sagawa et al., 2020) and underpin their notorious weakness to generalize out-of-distribution (e.g., Geirhos et al., 2020).”
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics (2022, background): “The recent interest has been particularly towards finding and dealing with minority group samples to promote model fairness (Sagawa et al., 2019; 2020; Liu et al., 2021; Zhang et al., 2022; Nam et al., 2022).”
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021, background): “Spurious correlations can also be mitigated by data-based, optimization-based, and post-processing methods (Zmigrod et al., 2019; Kaushik et al., 2020; Sagawa et al., 2020a; Yaghoobzadeh et al., 2021; Clark et al., 2019).”
Ethical Machine Learning in Healthcare (2020, background): “When data lack diversity and are not representative of the target population where the model would be deployed, overfitting algorithms to this data has the potential to disproportionately harm marginalized groups (112).”
Just Train Twice: Improving Group Robustness without Training Group Information (2021, methodology): “01 from Sagawa et al. (2020a), without tuning.”
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift (2022, background): “Our assumption that the model outputs are conditionally independent is weaker than assumptions in prior conceptual models of distribution shifts Chen et al. [2020], Sagawa et al. [2020b], Nagarajan et al. [2020] where robust and spurious features are disjoint f ens .”
The Risks of Invariant Risk Minimization (2020, unknown): “This model is natural and flexible; it generalizes several existing models used to analyze learning under the existence of adversarial distribution shift or non-invariant correlations (Schmidt et al., 2018; Sagawa et al., 2020).”
Better Language Model with Hypernym Class Prediction (2022, methodology): “This is inline with observations made recently in vision tasks (Sagawa et al., 2020).”
Causally motivated shortcut removal using auxiliary labels (2021, background): “This holds even if the foreground object alone is suﬃcient to achieve optimal predictive performance on the training distribution [Nagarajan et al., 2020, Sagawa et al., 2020a].”