Generalizing to Unseen Domains: A Survey on Domain Generalization

By Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin. 2021

1,790 citations recorded by the source corpus.

Machine learning systems generally assume that the training and testing distributions are the same. To this end, a key requirement is to develop models that can generalize to unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. First, we provide a formal definition of domain generalization and discuss several related fields. We then thoroughly review the theories related to domain generalization and carefully analyze the theory behind generalization. We categorize recent algorithms into three classes: data manipulation, representation learning, and learning strategy, and present several popular algorithms in detail for each category. Third, we introduce the commonly used datasets, applications, and our open-sourced codebase for fair evaluation. Finally, we summarize existing literature and present some potential research topics for the future.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Tao Qin
3. Jindong Wang
4. Percy Liang
5. Bo Li
6. Peng Cui
7. Quoc V. Le
8. Jianfeng Gao

In Computer Vision and Pattern Recognition

1. Tao Qin
2. Bo Li
3. Jindong Wang
4. Peng Cui
5. Bowen Zhou
6. Peng Qi
7. Nan Shao
8. Wenjie Feng

In Representation & Self-Supervised Learning

1. Jindong Wang
2. Peng Cui
3. Wenjie Feng
4. Zhilin Yang
5. Bo Li
6. Tao Qin
7. Yujun Chen
8. Nan Shao

Citations in the corpus

• ZeroPrompt: Scaling Prompt-Based Pretraining to 1, 000 Tasks Improves Zero-Shot Generalization (2022, background): “Domain generalization addresses the model’s ability to generalize to unseen test domains with data from several different but related domains (Wang et al., 2021).”
• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “For further information, we refer the reader to existing literature ( e.g. , [5, 52, 53]).”
• On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective (2023, background): “Out-of-distribution robustness On the other hand, OOD robustness (generalization) (Wang et al., 2022; Shen et al., 2021) aims to learn an optimal classifier on an unseen distribution by training on existing data.”
• Trustworthy AI: From Principles to Practices (2021, background): “This challenge arises from various practical AI tasks [343, 391] in the area of transfer learning [255, 350].”
• A Survey on Evaluation of Large Language Models (2023, background): “Li et al. [104] provided an overview of OOD evaluation for language models: adversarial robustness, domain generalization, and dataset biases.”
• AdaRNN: Adaptive Learning and Forecasting of Time Series (2021, background): “Similar to DA in general methodology, domain generalization (DG) [49] also learns a domain-invariant model on multiple source domains in the hope that it can generalize well to the target domain [2, 26, 30].”
• Teaching Models to Express Their Uncertainty in Words (2022, methodology): “To generalize calibration to a new target domain, methods often require samples from the target or from additional source domains (Gong et al., 2021; Csurka, 2017; Wang et al., 2021).”
• Towards Out-Of-Distribution Generalization: A Survey (2021, methodology): “Regarding to different methodological focuses, similar to Wang et al. [14], we divide DG methods into three branches, namely representation learning, training strategy and data augmentation.”
• Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift (2022, background): “Domain adaptation (Tzeng et al., 2017; Ganin et al., 2016; Wang et al., 2018) and domain generalization (Wang et al., 2021; Li et al., 2018; Muandet et al., 2013) are common ways to alleviate the distribution shift.”
• Single-Source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification (2022, neutral): “datasets) that generalize well on TD [16].”