FDA: Fourier Domain Adaptation for Semantic Segmentation

By Yanchao Yang, Stefano Soatto. 2020

1,162 citations recorded by the source corpus.

We describe a simple method for unsupervised domain adaptation, whereby the discrepancy between the source and target distributions is reduced by swapping the low-frequency spectrum of one with the other. We illustrate the method in semantic segmentation, where densely annotated images are aplenty in one domain (synthetic data), but difficult to obtain in another (real images). Current state-of-the-art methods are complex, some requiring adversarial optimization to render the backbone of a neural network invariant to the discrete domain selection variable. Our method does not require any training to perform the domain alignment, just a simple Fourier Transform and its inverse. Despite its simplicity, it achieves state-of-the-art performance in the current benchmarks, when integrated into a relatively standard semantic segmentation model. Our results indicate that even simple procedures can discount nuisance variability in the data that more sophisticated methods struggle to learn away.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, background): “A second class of UDA algorithms directly minimize suitable loss functions that enforce domain alignment (Wu et al. 2018; Zhang, David, and Gong 2017; Gabourie et al. 2019; Zhang et al. 2019; Lee et al. 2019; Yang and Soatto 2020).”
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “Others propose to conduct frequency-aware augmentation [85], [86] to improve the robustness or to solve the problem of domain adaptation.”
PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training (2021, methodology): “Finally, [56] performs image-to-image translation for UDA in frequency space rather than pixel space using a Fourier transform.”
Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation (2021, methodology): “The rationale behind this setting is that i) recent state-of-the-art UDA methods in semantic segmentation [42, 17, 48, 44, 31] share similar spirits with BDL, so conducting pilot studies on this method would be representative; ii) a black-box attack assumes that the attacker can only access very lim”
Phase Consistent Ecological Domain Adaptation (2020, background): “The concurrent work [43] shows that swapping the amplitude component of an image with one from the other domain preserves the semantic content while aligning the two domains.”
BAPA-Net: Boundary Adaptation and Prototype Alignment for Cross-domain Semantic Segmentation (2021, methodology): “The previous cross-domain semantic segmentation methods are mainly divided into three categories: 1) methods based on adversarial training including AdvEnt [52], BDL [30] and FADA [54], 2) methods based on selftraining including PyCDA [31], CRST [66], R-MRNet [64], UDADT [57] and IAST [41], and 3) m”
FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space (2021, background): “, style) and highlevel semantics of an image can be respectively captured by amplitude and phase spectrum in the frequency space, as revealed by visual psychophysics [13, 42, 57].”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, methodology): “For adversarial learning, numerous works focus on reducing the distribution misalignment in the image level [12, 13, 10, 14, 15, 5, 16], feature level [17, 8, 9], and output level [2–4 , 18].”
Global Filter Networks for Image Classification (2021, methodology): “With the breakthroughs of CNNs in vision [14, 13], there are a variety of works that start to incorporate Fourier transform in some deep learning method [24, 46, 9, 22, 6] for vision tasks.”
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, methodology): “Most of the UDA methods pre-compute the pseudolabels offline, train the model, and repeat the process [14, 88, 99, 100].”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “To put this into context, the training of other UDA methods [10, 24, 25] can take several days.”
A Fourier-based Framework for Domain Generalization (2021, methodology): “Note that the AM strategy is essentially different from the spectral transfer operation proposed in [48].”
Undoing the Damage of Label Shift for Cross-domain Semantic Segmentation (2022, methodology): “Recently, selfsupervised methods, including self-training [16, 17, 32, 49, 51, 52, 54, 55] and semantic consistency [26, 41, 47], have been shown to significantly improve the cross-domain semantic segmentation performance.”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “The second is image translation based which translates images from source domains to target domains to mitigate domain gaps [21, 58, 37, 80, 23, 32, 76, 75].”
Continual Test-Time Domain Adaptation (2022, background): “Alternatively, the alignment can also be done in the input space [18, 67].”
ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding (2021, methodology): “A multitude of recent works [7, 15, 22, 23, 26, 41, 43, 44, 46, 48, 51, 53, 59, 60, 62, 65, 66] have focused on unsupervised domain adaptation (UDA) for semantic segmentation, but most of them are validated only on an artiﬁcial synthetic-to-real setting, using GTA5 [34] and SYNTHIA [36] as source…”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, positive): “Besides, OST [470] proposes mapping the target input onto the source data manifold using Fourier style transfer [238], serving as a pre-processor to the primary network.”