Learning to Optimize Domain Specific Normalization for Domain Generalization
By Seonguk Seo, Yumin Suh, Dongwan Kim, Jongwoo Han, Bohyung Han. 2019
267 citations recorded by the source corpus.
We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains. Our approach employs multiple normalization methods while learning separate affine parameters per domain. For each domain, the activations are normalized by a weighted average of multiple normalization statistics. The normalization statistics are kept track of separately for each normalization type if necessary. Specifically, we employ batch and instance normalizations in our implementation to identify the best combination of these two normalization methods in each domain. The optimized normalization layers are effective to enhance the generalizability of the learned model. We demonstrate the state-of-the-art accuracy of our algorithm in the standard domain generalization benchmarks, as well as viability to further tasks such as multi-source domain adaptation and domain generalization in the presence of label noise.
Subfields: Artificial Intelligence, Civil and Structural Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Civil and Structural Engineering
In Computer Vision and Pattern Recognition
Citations in the corpus
- In Search of Lost Domain Generalization (2020, background): “This augmentation protocol is increasingly standard in state-of-the-art DG work (Nam et al., 2019; Huang et al., 2020; Krueger et al., 2020; Carlucci et al., 2019a; Zhou et al., 2020; Dou et al., 2019; Hendrycks et al., 2020; Wang et al., 2020a; Seo et al., 2020; Chattopadhyay et al., 2020).”
- Deep Stable Learning for Out-Of-Distribution Generalization (2021, background): “A common approach is to extract domain-invariant features over multiple source domains [17, 25, 32, 34, 40, 10, 22, 43, 48, 40] or to aggregate domain-specific modules [36, 37].”
- Domain Generalization with MixStyle (2021, background): “Some works explored domain-specific parameterization, such as domain-specific weight matrices (Li et al., 2017) and domain-specific BN (Seo et al., 2020).”
- A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, unknown): “DSON [277] proposes to fuse IN with BN statistics by linearly interpolating the means and variances, incorporating the instance-specific information in the trained model.”
- RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening (2021, background): “In general, most DG methods [27, 53, 10, 37, 38, 15, 25, 2, 26, 31] accomplish this through the learning of a shared representation across multiple source domains.”
- Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, unknown): “Literature Category Application [31] Augmentation Car detection [83] Augmentation, meta-learning Image classification [34] Augmentation Image classification [30] Augmentation Reinforcement learning [33] Augmentation Reinforcement learning CrossGrad [35] Augmentation, adversarial Font/Image/speech re”
- FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space (2021, background): “Domain generalization [5, 9, 12, 14, 43, 47, 58, 59] aims to learn a model from multiple source domains such that it can directly generalize to unseen target domains.”
- Domain Generalization: A Survey (2021, methodology): “The second is about the formulation of the distance function (e.g., using (cid:96) 2 [169] or softmax [171]).”