DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation

By Zuxuan Wu, Xintong Han, Yen-Liang Lin, M. Uzunbas, Tom Goldstein, Ser-Nam Lim, Larry S. Davis. 2018

273 citations recorded by the source corpus.

Harvesting dense pixel-level annotations to train deep neural networks for semantic segmentation is extremely expensive and unwieldy at scale. While learning from synthetic data where labels are readily available sounds promising, performance degrades significantly when testing on novel realistic data due to domain discrepancies. We present Dual Channel-wise Alignment Networks (DCAN), a simple yet effective approach to reduce domain shift at both pixel-level and feature-level. Exploring statistics in each channel of CNN feature maps, our framework performs channel-wise feature alignment, which preserves spatial structures and semantic information, in both an image generator and a segmentation network. In particular, given an image from the source domain and unlabeled samples from the target domain, the generator synthesizes new images on-the-fly to resemble samples from the target domain in appearance and the segmentation network further refines high-level features before predicting semantic maps, both of which leverage feature statistics of sampled images from the target domain. Unlike much recent and concurrent work relying on adversarial training, our framework is lightweight and easy to train. Extensive experiments on adapting models trained on synthetic segmentation benchmarks to real urban scenes demonstrate the effectiveness of the proposed framework.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “We have also included methods that are based on direct distributional matching which are more similar to MAS3: FCNs in the Wild (Hoffman et al. 2016), CDA (Zhang, David, and Gong 2017), DCAN (Wu et al. 2018), SWD (Lee et al. 2019), Cross-City (Chen et al. 2017b).”
Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation (2020, background): “To deal with the challenge, some pioneering works [Hoffman et al., 2018, Wu et al., 2018] propose to transfer the visual style of the source-domain data to the target domain.”
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation (2019, methodology): “evaluated on the Cityscapes validation set for 19 compatible classes, which match the same oracle performance reported in the recent literature [69, 74].”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, background): “Other approaches to aligning the source and target domains include channel-wise alignment of feature maps [41], done differently for ”stuff” classes and ”things” classes in [39], transfer of texture from target to source [18], and transfer of the low-frequency part of the spectrum in Fourier space f”
Unmanned Aerial Vehicle for Remote Sensing Applications - A Review (2019, background): “Minimizing domain shift in the pixel space also presents as a promising way to deal with the varying imaging conditions of UAVs-based RS [117].”
ESL: Entropy-guided Self-supervised Learning for Domain Adaptation in Semantic Segmentation (2020, methodology): “ to translate source images into target-like ones. The translated source images, with different visual styles yet preserved semantic labels, indeed bring adaptation effects for models trained on them [8,25]. In this work, we base our iterative self-supervised framework on top of the state-of-the-art”
Enhancing Photorealism Enhancement (2021, unknown): “and real images [17], [18], [19], [20], [21], [22], [23], [24], [25].”
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation (2020, background): “[76] explored channel-wise statistics alignment of CNN features to guide the generation process.”
Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation (2021, background): “…learning [53] is ﬁrst introduced by Hoff-man et al. to perform unsupervised domain adaptation task [10], [14], [15], [52], large quantities of competing variants [13], [27], [41], [42], [43], [46] relying on adversarial framework have been developed to explore shared domain-invariant features.”
Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation (2019, methodology): “We compare our proposed method with Curriculum DA [51], CyCADA [14], MCD [39], LSD-seg [41], AdaptSegNet [44], ROAD [5], Conservative Loss [55], DCAN [47], and CBST [57].”
Confidence Regularized Self-Training (2019, methodology): “Examples of these methods include maximum mean discrepancy (MMD) [33, 62], deep correlation alignment (CORAL) [56], sliced Wasserstein discrepancy [28], adversarial learning at input-level [16, 24, 68], feature level [8, 14, 23, 31, 50, 61, 64], output space level [60], and a variety of follow up wo”
Phase Consistent Ecological Domain Adaptation (2020, methodology): “[41] reduces domain shift at both image and feature levels by aligning statistics in each channel of CNN feature maps in order to preserve spatial structures.”
University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization (2020, methodology): “In light of the recent practice using synthetic training data [14, 17, 26, 37], we propose a multi-view multi-source dataset called University-1652, containing synthetic drone-view images.”