PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training

By Luke Melas-Kyriazi, Arjun K. Manrai. 2021

139 citations recorded by the source corpus.

Unsupervised domain adaptation is a promising technique for semantic segmentation and other computer vision tasks for which large-scale data annotation is costly and time-consuming. In semantic segmentation, it is attractive to train models on annotated images from a simulated (source) domain and deploy them on real (target) domains. In this work, we present a novel framework for unsupervised domain adaptation based on the notion of target-domain consistency training. Intuitively, our work is based on the idea that in order to perform well on the target domain, a model’s output should be consistent with respect to small perturbations of inputs in the target domain. Specifically, we introduce a new loss term to enforce pixelwise consistency between the model’s predictions on a target image and a perturbed version of the same image. In comparison to popular adversarial adaptation methods, our approach is simpler, easier to implement, and more memory-efficient during training. Experiments and extensive ablation studies demonstrate that our simple approach achieves remarkably strong results on two challenging synthetic-to-real benchmarks, GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dengxin Dai
2. Luc Van Gool
3. Lukas Hoyer
4. Mingkui Tan
5. Chuang Gan
6. Shuaicheng Niu
7. Yifan Zhang
8. Wayne Zhang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Mingkui Tan
3. Dengxin Dai
4. Chuang Gan
5. Lukas Hoyer
6. Wayne Zhang
7. Yifan Zhang
8. Ming-Hsuan Yang

In Radiology, Nuclear Medicine and Imaging

1. Lukas Hoyer
2. Luc Van Gool
3. Dengxin Dai
4. Shuaicheng Niu
5. Chuang Gan
6. Yifan Zhang
7. Mingkui Tan

Citations in the corpus

• Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation (2022, background): “Step 3 [20,21,22,23,24,25,26,27,28,29] Postit Notes, Mug, Webcam, Desk Lamp, Telephone, Helmet, Mouse, Pen, Monitor, Mop”
• DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, methodology): “In order to avoid training instabilities, pseudo-label prototypes [88] or consistency regularization [61,65] based on data augmentation [1,9,50] or domain-mixup [67, 91] are used.”
• HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, unknown): “In order to regularize the training and to avoid pseudo-label drift, approaches such as confidence thresholding [43, 92], pseudo-label prototypes [39, 80, 81], and consistency regularization [50, 52, 57] based on data augmentation [1, 10, 44, 46], different context [35, 89], domain-mixup [20, 29, 39”
• Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation (2022, methodology): “Apart from semi-supervised classification, the methodology in FixMatch has swept across a wide range of research topics and achieved booming success, such as semantic segmentation [19, 28, 81], object detection [42, 57, 66], unsupervised domain adaptation [45], and action recognition [62,67].”