Out of Distribution Generalization in Machine Learning

By Martín Arjovsky. 2021

116 citations recorded by the source corpus.

Machine learning has achieved tremendous success in a variety of domains in recent years. However, a lot of these success stories have been in places where the training and the testing distributions are extremely similar to each other. In everyday situations when models are tested in slightly different data than they were trained on, ML algorithms can fail spectacularly. This research attempts to formally define this problem, what sets of assumptions are reasonable to make in our data and what kind of guarantees we hope to obtain from them. Then, we focus on a certain class of out of distribution problems, their assumptions, and introduce simple algorithms that follow from these assumptions that are able to provide more reliable generalization. A central topic in the thesis is the strong link between discovering the causal structure of the data, finding features that are reliable (when using them to predict) regardless of their context, and out of distribution generalization.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers (2021, background): “However, several important issues remain unsolved, including the ability to generalize well to novel, out-of-distribution data (Arjovsky, 2021).”
Objective Robustness in Deep Reinforcement Learning (2021, background): “It is widely known that deep learning systems may fail in unexpected ways when deployed out-of-distribution; for example, a classifier trained on images of cows in Alpine pastures will typically fail to recognize an image of a cow that is taken on a sandy beach [7; 4].”
Attention, please! A survey of neural attention models in deep learning (2021, unknown): “2020) (Arjovsky 2020).”
Text Data Augmentation for Deep Learning (2021, background): “As Arjvosky writes, “if the test data is arbitrary or unrelated to the training data, then generalization is obviously futile” [86].”