Domain Generalization: A Survey

By Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao A. Xiang, Chen Change Loy. 2021

1,606 citations recorded by the source corpus.

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce. This is because most learning algorithms strongly rely on the i.i.d. assumption on source/target data, which is often violated in practice due to domain shift. Domain generalization (DG) aims to achieve OOD generalization by using only source data for model learning. Over the last ten years, research in DG has made great progress, leading to a broad spectrum of methodologies, e.g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, to name a few; DG has also been studied in various application areas including computer vision, speech recognition, natural language processing, medical imaging, and reinforcement learning. In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. Specifically, we first cover the background by formally defining DG and relating it to other relevant fields like domain adaptation and transfer learning. Then, we conduct a thorough review into existing methods and theories. Finally, we conclude this survey with insights and discussions on future research directions.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Learning to Prompt for Vision-Language Models (2021, neutral): “Since CoOp requires training on a speciﬁc data distribution, it risks learning spurious correlations that are detrimental to generalization in unseen distributions (domains), as suggested in recent studies (Taori et al., 2020; Zhou et al., 2021).”
Conditional Prompt Learning for Vision-Language Models (2022, background): “Generalization to out-of-distribution data is a capability essential for machine learning models to succeed in practical applications [47, 62].”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “Domain Adaptation/Generalization Domain Adaptation (DA) (Wang & Deng, 2018) and Domain Generalization (DG) (Zhou et al., 2021b) also follow “open-world” assumption.”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, neutral): “This issue arises because the learner, during training, operates on the states generated by the expert policy, yet during testing, it deals with states induced by its actions [15].”
On the Importance of Gradients for Detecting Distributional Shifts in the Wild (2021, background): “However, covariate shifts are commonly used to evaluate model robustness [12] and domain generalization performance [52], where the label space Y remains the same during test time.”
Towards Out-Of-Distribution Generalization: A Survey (2021, methodology): “Here we brieﬂy introduce the three branches of methods, and one can refer to existing surveys of this ﬁeld [14], [15] for more details of DG methods.”
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, background): “While the neighborhood OOD generalization community focuses on ensuring the robustness of models to maintain high performance on OOD samples with domain shift [8], OOD detection, on the other hand, emphasizes the model reliability by requiring the identification of samples with semantic shift [5].”
Trustworthy AI: From Principles to Practices (2021, methodology): “A summary of commonly used datasets and benchmarks for domain generalization can be found in Reference [391] and covers the tasks of object recognition, action recognition, segmentation, and face recognition.”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “• Human-in-the-loop-based systems should have high robustness and generalization ability to domain changes, disturbances, and “out-of-range” samples [161].”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “[3] also wrote a survey on DG, while their focus is on DG in computer vision field.”
Semantically Coherent Out-of-Distribution Detection (2021, background): “training [3]; 2) they tend to recklessly classify a test image into a certain training class, even though it has a semantic shift from training, that is, it may not belong to any training class [4].”
Domain Generalization with MixStyle (2021, background): “Domain generalization (DG) aims to address such a problem (Zhou et al., 2021; Blanchard et al., 2011; Muandet et al., 2013; Li et al., 2018a; Zhou et al., 2020b; Balaji et al., 2018; Dou et al., 2019; Carlucci et al., 2019).”
Federated learning for medical image analysis: A survey (2023, background): “An interesting question is: when facing data from unseen sites which are outside of a federation (outside clients), how to guarantee the generalizability of an FL model? This is typically a domain generalization problem (Zhou et al., 2023; Wang et al., 2022) or a test-time adaptation problem (i.”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “For instance, domain generalization (DG) [5] aims to learn a model using data from one or multiple source domains that can generalize well to any out-of-distribution target domain.”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “performance collapse of the model [48], [49], [50].”
Domain Generalization Under Posterior Drift (2025, unknown): “• Zhou et al. [2023] (survey): “The goal of DG is to learn a predictive model f : X → Y using only source domain data such that the prediction error on an unseen domain T = (cid:8) x T (cid:9) is minimized.””