Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime

By Dengxin Dai, Luc Van Gool. 2018

295 citations recorded by the source corpus.

This work addresses the problem of semantic image segmentation of nighttime scenes. Although considerable progress has been made in semantic image segmentation, it is mainly related to daytime scenarios. This paper proposes a novel method to progressive adapt the semantic models trained on daytime scenes, along with large-scale annotations therein, to nighttime scenes via the bridge of twilight time — the time between dawn and sunrise, or between sunset and dusk. The goal of the method is to alleviate the cost of human annotation for nighttime images by transferring knowledge from standard daytime conditions. In addition to the method, a new dataset of road scenes is compiled; it consists of 35,000 images ranging from daytime to twilight time and to nighttime. Also, a subset of the nighttime images are densely annotated for method evaluation. Our experiments show that our method is effective for knowledge transfer from daytime scenes to nighttime scenes, without human annotation.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Revisiting Self-Supervised Visual Representation Learning (2019, methodology): “a model trained on the ImageNet (ILSVRC-2012) dataset [38] can only recognize 1000 semantic categories or a model that was trained to perceive road traffic at daylight may not work in darkness [5, 4].”
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, background): “Evaluating robustness to environmental changes in autonomous driving In recent years, weather conditions turned out to be a central limitation for state-of-the art autonomous driving systems [Sakaridis et al., 2018a, Volk et al., 2019, Dai and Van Gool, 2018, Chen et al., 2018, Lee et al., 2018].”
SWAD: Domain Generalization by Seeking Flat Minima (2021, background): “For example, a self-driving car should adapt to adverse weather or day-to-night shifts [1, 2].”
In Search of Lost Domain Generalization (2020, background): “For instance, self-driving car systems struggle to perform under conditions different to those of training, including variations in light [Dai and Van Gool, 2018], weather [Volk et al., 2019], and object poses [Alcorn et al., 2019].”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, methodology): “We evaluate GCMA on Dark Zurich-test against the state-of-the-art adaptation approaches AdaptSegNet [31] and DMAda [8] and report standard IoU performance in Table 3, including invalid pixels which are assigned a le- gitimate semantic label in the evaluation.”
End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners (2018, background): “Autonomous driving has seen dramatic advances in recent years, for instance for road scene parsing [23,67,79,24], lane following [46,37,17], path planning [12,18,62,63], and end-to-end driving models [77,22,21,56].”
Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset (2019, background): “Recently, deep learning has achieved great success in the field of low-level image processing [66] and nighttime scenes modeling [57] and understanding [18,62].”
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization (2021, background): “Accuracy suffers when the model performs under conditions different to those of training, such as variations in light [8], weather [51], object poses [2], textures [16], or object backgrounds [5].”
CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency (2019, methodology): “Recent advances include applying class-wise adversarial learning [4] or leveraging self-paced learning policy [53] for adapting synthetic-to-real or cross-city adaptation [4], adopting curriculum learning for synthetic-to-real foggy scene adaptation [34], or progressively adapting models from daytim”
ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding (2021, background): “CS: Cityscapes [8], FC: Foggy Cityscapes [38], FC-DBF: Foggy Cityscapes-DBF [37], FZ: Foggy Zurich [37], ND: Nighttime Driving [10], DZ: Dark Zurich [40].”
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, background): “curriculum [9, 13, 14, 109], exploit the increased domainrobustness of Transformers [34, 35, 83, 102], align the domains with contrastive learning [38, 97], use graph matching [4, 49, 50], or adapt multi-resolution inputs [35].”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, unknown): “Several works also combine self-training and adversarial training [34, 37, 64, 86], minimize the entropy [7, 62, 63], refine boundaries [39], use a curriculum [12, 13, 82], or learn auxiliary tasks [8, 29, 63, 66].”
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding (2018, background): “The concurrent work in [13] on adaptation of semantic models from daytime to nighttime solely with real data is closely related to ours.”
Benchmarking Low-Light Image Enhancement and Beyond (2021, background): “2013a) 85 Real No VV2 24 Real No ExDARK (Loh and Chan 2019) 7363 Real Object bounding boxes/categories Nighttime driving (Dai and Gool 2018) 50 (35,000 unlabeled) Real Densely Annotated Semantic Labels Dark Zurich (Sakaridis et al.”