ACE: Adapting to Changing Environments for Semantic Segmentation

By Zuxuan Wu, Xin Wang, Joseph E. Gonzalez, Tom Goldstein, Larry S. Davis. 2019

109 citations recorded by the source corpus.

Deep neural networks exhibit exceptional accuracy when they are trained and tested on the same data distributions. However, neural classifiers are often extremely brittle when confronted with domain shift---changes in the input distribution that occur over time. We present ACE, a framework for semantic segmentation that dynamically adapts to changing environments over time. By aligning the distribution of labeled training data from the original source domain with the distribution of incoming data in a shifted domain, ACE synthesizes labeled training data for environments as it sees them. This stylized data is then used to update a segmentation model so that it performs well in new environments. To avoid forgetting knowledge from past environments, we introduce a memory that stores feature statistics from previously seen domains. These statistics can be used to replay images in any of the previously observed domains, thus preventing catastrophic forgetting. In addition to standard batch training using stochastic gradient decent (SGD), we also experiment with fast adaptation methods based on adaptive meta-learning. Extensive experiments are conducted on two datasets from SYNTHIA, the results demonstrate the effectiveness of the proposed approach when adapting to a number of tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[193] proposed to store feature statistics of previous images to help train the image generator.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…to perform continual semantic segmentation of medical images (Ozdemir, Fuernstahl, & Goksel, 2018; Ozdemir & Goksel, 2019), remote sensing data (Tasar, Tarabalka, & Alliez, 2019; Wu, Wang et al., 2019), and on standard computer vision benchmarks (Cermelli, Mancini, Bulo, Ricci, & Caputo, 2020).”
Open Compound Domain Adaptation (2019, background): “The goal is to extend unsupervised domain adaptation to latent [16, 51, 32] or multiple [11, 9, 54] or continuous [2, 13, 31, 50] target domains, when only the source domain has class labels.”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, background): “For adversarial learning, numerous works focus on reducing the distribution misalignment in the image level [12, 13, 10, 14, 15, 5, 16], feature level [17, 8, 9], and output level [2–4 , 18].”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, background): “This can be targeted at different levels such as at the pixel level [29, 30, 31], feature map level [32, 11, 33] or semantic level [14, 34].”
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, background): “As suggested by the theoretical analysis [3], domain alignment methods focus on reducing the distribution mismatch by optimizing some divergence [30, 35] or adopting adversarial training [20, 40] at either the image level [1, 12, 19, 26, 49, 63, 68], the intermediate feature level [6, 27, 48, 58] or”