Distributionally Robust Neural Networks

By Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, Percy Liang. 2020

768 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “…techniques from causal inference (Schölkopf, 2019), uncertainty estimation (Gal and Ghahramani, 2016; Kendall and Gal, 2017), density estimation and generative modeling (Kingma et al., 2014), distributional robustness (Sinha et al., 2017; Sagawa et al., 2019) and invariance (Arjovsky et al., 2019).”
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2021, background): “and distributionally robust optimization (Oren et al., 2019; Sagawa et al., 2020), where the goal is to perform well over a family of distributions.”
Self-supervised learning in medicine and healthcare (2022, background): “This is commonly seen in models that rely on image backgrounds more heavily to predict the subject 51 .”
Challenges and Strategies in Cross-Cultural NLP (2022, methodology): “Most methods to balance for bias require access to protected demographic attributes (Sagawa et al., 2020), and these only partially reﬂect culture.”
Algorithm fairness in artificial intelligence for medicine and healthcare (2023, methodology): “…dataset shift occurs when there is a mismatch between the distributions of the training and test datasets during algorithm development, (that is, P train ( X ) ≠ P test ( X ) and P train ( Y ) ≠ P test ( Y ) ), and may lead to disparate performance at the subgroup level 86,107,119,122,123 .”
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (2024, neutral): “…a large literature on mitigating robustness to spurious correlations, including techniques which rely on directly optimizing worst-group accuracy (Sagawa et al., 2020; Oren et al., 2019; Zhang et al., 2021; Sohoni et al., 2022; Nam et al., 2022), automatically or manually reweighting data…”
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks (2020, methodology): “Distributional robustness considers small adversarial perturbations of the data distribution, and ensures that the model performs well under these (Goh & Sim, 2010; Sagawa et al., 2020; Sinha et al., 2018; Staib & Jegelka, 2019).”