Domain Generalization Using a Mixture of Multiple Latent Domains

By Toshihiko Matsuura, Tatsuya Harada. AAAI Conference on Artificial Intelligence, 2019

357 citations recorded by the source corpus.

When domains, which represent underlying data distributions, vary during training and testing processes, deep neural networks suffer a drop in their performance. Domain generalization allows improvements in the generalization performance for unseen target domains by using multiple source domains. Conventional methods assume that the domain to which each sample belongs is known in training. However, many datasets, such as those collected via web crawling, contain a mixture of multiple latent domains, in which the domain of each sample is unknown. This paper introduces domain generalization using a mixture of multiple latent domains as a novel and more realistic scenario, where we try to train a domain-generalized model without using domain labels. To address this scenario, we propose a method that iteratively divides samples into latent domains via clustering, and which trains the domain-invariant feature extractor shared among the divided latent domains via adversarial learning. We assume that the latent domain of images is reflected in their style, and thus, utilize style features for clustering. By using these features, our proposed method successfully discovers latent domains and achieves domain generalization even if the domain labels are not given. Experiments show that our proposed method can train a domain-generalized model without using domain labels. Moreover, it outperforms conventional domain generalization methods, including those that utilize domain labels.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/41a4aa801da162ca777c14243e8e3a14e9bc2d45

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, neutral): “Literature Category Application [26] Augmentation Car detection [77] Augmentation, meta-learning Image classification [29] Augmentation Image classification [25] Augmentation Reinforcement learning [28] Augmentation Reinforcement learning CrossGrad [30] Augmentation, adversarial Font/Image/speech re”
Deep Stable Learning for Out-Of-Distribution Generalization (2021, methodology): “M-ADA and DGMMLD fail to outperform ResNet-18 on NICO under this setting.”
Reducing Domain Gap via Style-Agnostic Networks (2019, methodology): “We evaluate the efficacy of SagNets against state-of-the-art DG methods including D-SAM [9], JiGen [5], Epi-FCR [31], MASF [8], and MMLD [37].”
Domain Generalization using Causal Matching (2020, neutral): “(Carlucci et al., 2019), S-MLDG (Li et al., 2020), D-SAM (D’Innocente & Caputo, 2018), MMLD (Matsuura & Harada, 2020), DDAIG (Zhou et al., 2020) SagNet (Nam et al., 2019), DDEC (Asadi et al., 2019), DANN (Ganin et al., 2016), C-DANN (Li et al., 2018d), DRO (Sagawa et al., 2019), Mixup (Xu et al., 20”
Learning to Diversify for Single Domain Generalization (2021, methodology): “AGG [25], HEX [44], PAR [43], JiGen [5], ADA [42], MEADA [49], MMLD [31] and our method belong to the latter category.”
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization (2021, methodology): “(Muandet et al., 2013; Zhao et al., 2019; Albuquerque et al., 2019; Piratla et al., 2020; Matsuura and Harada, 2020; Deng et al., 2020; Pagnoni et al., 2018; Greenfeld and Shalit, 2020; Garg et al., 2021) is an incomplete representative list of works that build the theory of domain adaptation and…”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, background): “Domain generalization further removes the requirement of unlabeled target domain data in multi-source UDA (Matsuura and Harada, 2020).”
FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space (2021, methodology): “There are other methods tackling DG by manipulating deep neural network architectures [19, 23, 35], leveraging self-supervision signals [3, 54], designing training heuristics [17, 25], or conducting data augmentations [48, 53, 60, 61], which are free from requirement of data centralization.”
In Search of Lost Domain Generalization (2020, background): “Matsuura and Harada [2019] leverages clustering techniques to learn domain-invariant features even when the separation between training domains is not given.”
Domain Generalization: A Survey (2021, methodology): “- Domain-Adversarial Learning [28], [39], [59], [62], [145], [154], [155], [156], [157], [158]”
Learning to Optimize Domain Specific Normalization for Domain Generalization (2019, methodology): “The proposed algorithm is compared with several existing methods, which include JiGen [4], D-SAM [6], Epi-FCR [14], MetaReg [2], MASF [7], and MMLD [18].”