No More Discrimination: Cross City Adaptation of Road Scene Segmenters

By Yi-Hsin Chen, Wei-Yu Chen, Yu-Ting Chen, Bo-Cheng Tsai, Yu-Chiang Frank Wang, Min Sun. 2017

359 citations recorded by the source corpus.

Despite the recent success of deep-learning based semantic segmentation, deploying a pre-trained road scene segmenter to a city whose images are not presented in the training set would not achieve satisfactory performance due to dataset biases. Instead of collecting a large number of annotated images of each city of interest to train or refine the segmenter, we propose an unsupervised learning approach to adapt road scene segmenters across different cities. By utilizing Google Street View and its timemachine feature, we can collect unannotated images for each road scene at different times, so that the associated static-object priors can be extracted accordingly. By advancing a joint global and class-specific domain adversarial learning framework, adaptation of pre-trained segmenters to that city can be achieved without the need of any user annotation or interaction. We show that our method improves the performance of semantic segmentation in multiple cities across continents, while it performs favorably against state-of-the-art approaches requiring annotated training data.

Subfields: Artificial Intelligence, Civil and Structural Engineering, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Civil and Structural Engineering

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning To Simulate (2018, background): “Many recent works (Ganin et al., 2016; Chen et al., 2017; Tsai et al., 2018) focus on bridging this domain gap, particularly for computer vision tasks.”
Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “We have also included methods that are based on direct distributional matching which are more similar to MAS3: FCNs in the Wild (Hoffman et al. 2016), CDA (Zhang, David, and Gong 2017), DCAN (Wu et al. 2018), SWD (Lee et al. 2019), Cross-City (Chen et al. 2017b).”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, background): “ulti-source model through concatenating all available labeled data and learning a single large model and then transfers to a sparsely labeled target domain through distillation (Hinton et al., 2015). Chen et al. (2017) use an adversarial objective to align both global and class-speciﬁc statistics, w”
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (2018, methodology): “A similar approach of global and class-wise alignment is used in [5] with the class-wise alignment being done using adversarial training on grid-wise soft pseudo-labels.”
Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “DANN [20] MADA [13] CDAN [14] DANN-CA [17] RCA [16] IDDA [15] NOUN PRONOUN Concerning the above works, the conditional domain adversarial training methods [13]–[18], [20] are the most related to ours.”
Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation (2021, methodology): “NTHU [5] is composed of 4 real-world urban scenes data-sets, including Rio, Rome, Taipei and Tokyo, where each dataset consists of the training and testing subsets, and is ﬁnely annotated with 13 different object categories.”
Universal Source-Free Domain Adaptation (2020, background): “A recognition model trained on a certain distribution of labeled samples (source domain) often fails to generalize [7]”
Learning Semantic Segmentation From Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach (2018, methodology): “To facilitate the comparison with other state-of-the-art works, we further evaluate the proposed method on SYNTHIA to Cityscapes setting following [21, 57, 6, 5, 53, 49, 62].”
Domain Adaptation for Semantic Segmentation With Maximum Squares Loss (2019, methodology): “Rio Cross city [5] 74.”
Effective Use of Synthetic Data for Urban Scene Semantic Segmentation (2018, methodology): “This problem is due to the domain shift between real and synthetic data, and has thus been tackled by domain adaptation methods [6, 19, 20, 51, 27, 5], which, in essence, aim to reduce the gap between the feature distributions of the two domains.”
Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation (2019, methodology): “These results verify the Note that the IoUs of wall, fence, and pole in CCA (Chen et al. 2017) are not reported. effectiveness of the proposed method for learning domain-invariant features.”
Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation (2018, neutral): “4% and [6] by 3 .”
CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency (2019, methodology): “In addition to the synthetic-to-real adaptation, we conduct an experiment on the Cross-City dataset [4] which is a real-to-real adaptation.”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, methodology): “We demonstrate the efﬁcacy of the proposed method on the standard adaptation tasks of GTA5 [6] → Cityscapes [22], SYNTHIA [7] → Cityscapes and Cityscapes → NTHU Cross-City [38].”
Conditional Adversarial Domain Adaptation (2017, background): “This is different from aligning the features and classes separately [23, 8, 50].”