DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

By Lukas Hoyer, Dengxin Dai, Luc Van Gool. Computer Vision and Pattern Recognition, 2021

618 citations recorded by the source corpus.

As acquiring pixel-wise annotations of real-world images for semantic segmentation is a costly process, a model can instead be trained with more accessible synthetic data and adapted to real images without requiring their annotations. This process is studied in unsupervised domain adaptation (UDA). Even though a large number of methods propose new adaptation strategies, they are mostly based on outdated network architectures. As the influence of recent network architectures has not been systematically studied, we first benchmark different network architectures for UDA and newly reveal the potential of Transformers for UDA semantic segmentation. Based on the findings, we propose a novel UDA method, DAFormer. The network architecture of DAFormer consists of a Transformer encoder and a multi-level context-aware feature fusion decoder. It is enabled by three simple but crucial training strategies to stabilize the training and to avoid overfitting to the source domain: While (1) Rare Class Sampling on the source domain improves the quality of the pseudo-labels by mitigating the confirmation bias of self-training toward common classes, (2) a Thing-Class ImageNet Feature Distance and (3) a learning rate warmup promote feature transfer from ImageNet pretraining. DAFormer represents a major advance in UDA. It improves the state of the art by 10.8 mIoU for GTA→Cityscapes and 5.4 mIoU for Syruhia→Cityscapes and enables learning even difficult classes such as train, bus, and truck well. The implementation is available at https://github.com/lhoyer/DAFormer.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f6282d4bbd942f7f08ceb549cb4ee647a22aaa50

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “The comparably low performance is probably inherited from DAFormer [7], which is the basis of HRDA.”
Continual Test-Time Domain Adaptation (2022, neutral): “In recent years, self-training has also shown promising results by iteratively using gradually-improving target pseudo-labels to train the network [19, 36, 62, 75].”
Transformer-Based Visual Segmentation: A Survey (2023, background): “DAFormer [291] replaces the outdated backbone with the advanced transformer backbone [123] and Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Mapping (2022, methodology): “We adopted a metric-based method (MCD [50]), adversarial training methods (AdaptSeg [49], category-level adversarial network (CLAN) [32], TransNorm [55], and finegrained adversarial learning framework for domain adaptive (FADA) [52]), and self-training methods (pyramid curriculum DA (PyCDA) [24], cl”