ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

By Tuan-Hung Vu, Himalaya Jain, Max Bucher, Matthieu Cord, Patrick Pérez. 2018

1,529 citations recorded by the source corpus.

Semantic segmentation is a key problem for many computer vision tasks. While approaches based on convolutional neural networks constantly break new records on different benchmarks, generalizing well to diverse testing environments remains a major challenge. In numerous real-world applications, there is indeed a large gap between data distributions in train and test domains, which results in severe performance loss at run-time. In this work, we address the task of unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions. To this end, we propose two novel, complementary methods using (i) entropy loss and (ii) adversarial loss respectively. We demonstrate state-of-the-art performance in semantic segmentation on two challenging “synthetic-2-real” set-ups and show that the approach can also be used for detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “Finally, following the success of recent works exploiting entropy minimization [10] techniques, an alternative strategy could be to minimize the entropy of the latent representations opportunely preceded by L1 or softmax normalization of each feature vector in order to obtain a probability distribut”
PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, background): “Pseudo-labeling [39] is commonly used in domain adaptation for semantic segmentation [63, 41, 79, 58], where a model is trained on the union of real labels of a source dataset and pseudo labels assigned to an unlabeled target dataset.”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, background): “In domain adaptation [41, 57, 51, 10], several works aimed at translating a labeled image dataset into another domain, in which image annotation is either expensive or missing entirely, by leveraging image-to-image translation techniques.”
Unsupervised Intra-Domain Adaptation for Semantic Segmentation Through Self-Supervision (2020, methodology): “For instance, [28] proposes minimizing the target entropy value of the model outputs directly or using adversarial learning [25, 11] to close the domain gap for semantic segmentation.”
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, methodology): “In adversarial training, a learned domain discriminator provides supervision in a GAN framework [26] to encourage domain-invariant inputs [25, 31], features [21, 32, 58, 65, 86] or outputs [62, 74, 86, 89].”
Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation (2021, methodology): “…learning [53] is ﬁrst introduced by Hoff-man et al. to perform unsupervised domain adaptation task [10], [14], [15], [52], large quantities of competing variants [13], [27], [41], [42], [43], [46] relying on adversarial framework have been developed to explore shared domain-invariant features.”
Confidence Regularized Self-Training (2019, neutral): “4 AdvEnt [63] DeepLabv2 89.”
BAPA-Net: Boundary Adaptation and Prototype Alignment for Cross-domain Semantic Segmentation (2021, methodology): “Following the common experimental setup for crossdomain semantic segmentation [50, 37, 38, 51, 52, 61, 66, 64], we evaluate our method on two synthetic-to-real scenarios, namely GTA5 to Cityscapes and SYNTHIA to Cityscapes.”
Phase Consistent Ecological Domain Adaptation (2020, methodology): “[40] approaches UDA for semantic segmentation by entropy minimization of the pixel-wise predictions.”
Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentation (2023, neutral): “We compared BCP with V-Net [19], DAN [44], ADVNET [26], UA-MT [41], SASSNet [12], DTC [16] and CoraNet [22] in Table 2.”
Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation (2019, background): “The ideas of minimizing the entropy (uncertainty) of the output [39] or discrepancies between the outputs of two classiﬁers (voters) [26] have also been exploited to implicitly enforce category-level alignment.”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, methodology): “The first is adversarial training based which employs adversarial learning to align source and target distributions in the feature, output or latent space [22, 39, 69, 40, 66, 7, 81, 54, 55, 72, 1, 68, 26, 18, 25, 31].”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, methodology): “In addition, we compare our method with previous state-of-the-art works, including source data-dependent methods [2, 4, 23, 42–44, 5] and source data-free data-free methods [20, 19].”
RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening (2021, methodology): “Domain adaptation (DA) is an approach to mitigate the performance degradation caused by such a domain gap [3, 12, 18, 11, 69, 41, 60, 43, 52].”
Semi-supervised Medical Image Segmentation through Dual-task Consistency (2020, methodology): “Compared with Entropy Mini and fully supervised baseline, our method achieved better accuracy with comparable computational cost.”
Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification (2020, neutral): “ADVENT [41] 89.”