Bidirectional Learning for Domain Adaptation of Semantic Segmentation

By Yunsheng Li, Lu Yuan, Nuno Vasconcelos. 2019

688 citations recorded by the source corpus.

Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other.Furthermore, we propose a self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model. Experiments show that our method superior to the state-of-the-art methods in domain adaptation of segmentation with a big margin. The source code is available at https://github.com/liyunsheng13/BDL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, background): “Pseudo-labeling [39] is commonly used in domain adaptation for semantic segmentation [63, 41, 79, 58], where a model is trained on the union of real labels of a source dataset and pseudo labels assigned to an unlabeled target dataset.”
Hierarchical Multi-Scale Attention for Semantic Segmentation (2020, background): “[2], other semi-supervised self-training in semantic segmentation [30, 31, 32, 33, 34], and other approaches based on pseudo label such as [4, 35, 36, 3].”
What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation (2020, background): “2) Existing transfer models [16, 26, 27, 36, 38] perform worse than our model, since they pay equal attention to all semantic representation instead of neglecting irrelevant knowledge, which causes the negative transfer of untransferable knowledge.”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, background): “A major class of adaptation approaches, including [5,12,13,19,28,31,34,36,43,46], involves adversarial confusion or feature alignment between domains.”
MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation (2021, neutral): “The general idea of UDA semantic segmentation methods is to perform domain alignment through adversarial learning [2, 6, 7, 9, 11, 20, 22, 26, 40, 41] or utilize self-training strategy on target samples to adapt the segmentation models [4, 17, 23, 28, 30, 35, 46, 48, 49, 50, 51].”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “Several works also combine self-training and adversarial training [35,39,69,93], minimize the entropy [7,67,68], refine boundaries [41], use a curriculum [13,14,89], or learn auxiliary tasks [8,30,68,71].”
Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation (2021, methodology): “To fill this gap and to validate our hypothesis, we perform black-box attacks on BDL [18] by conducting the following two steps: 1) for each clean image in the test data, we first generate its adversarial example by attacking PSPNet [52] with ε = 0.”
ESL: Entropy-guided Self-supervised Learning for Domain Adaptation in Semantic Segmentation (2020, methodology): “This strategy has already been used in previous state-of-the-art paper in unsupervised domain adaptation for semantic segmentation [14].”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “The second is image translation based which translates images from source domains to target domains to mitigate domain gaps [21, 58, 37, 80, 23, 32, 76, 75].”
BAPA-Net: Boundary Adaptation and Prototype Alignment for Cross-domain Semantic Segmentation (2021, methodology): “The previous cross-domain semantic segmentation methods are mainly divided into three categories: 1) methods based on adversarial training including AdvEnt [52], BDL [30] and FADA [54], 2) methods based on selftraining including PyCDA [31], CRST [66], R-MRNet [64], UDADT [57] and IAST [41], and 3) m”
PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training (2021, background): “[38] trains a GANbased image-to-image translation network and a semantic segmentation network with the same backbone, and [32] extends the idea of cross-domain cycle-consistency to the multi-source setting.”
Light Field Super-Resolution with Zero-Shot Learning (2021, background): “, image classification [9, 20], semantic segmentation [17] and person re-identification [19].”
Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation (2021, methodology): “2) Different from the previous version (ELS-DA [12]) and BDL [26], our two complementary modules ( T A ð(cid:2)Þ and T R ð(cid:2)Þ ) could highlight where and how to capture trans-ferable knowledge across domains by incorporating with the knowledge aggregation-induced transferability perception.”