Seeing Motion in the Dark

By Qifeng Chen, Minh N. Do, Vladlen Koltun. IEEE International Conference on Computer Vision, 2019

330 citations recorded by the source corpus.

Deep learning has recently been applied with impressive results to extreme low-light imaging. Despite the success of single-image processing, extreme low-light video processing is still intractable due to the difficulty of collecting raw video data with corresponding ground truth. Collecting long-exposure ground truth, as was done for single-image processing, is not feasible for dynamic scenes. In this paper, we present deep processing of very dark raw videos: on the order of one lux of illuminance. To support this line of work, we collect a new dataset of raw low-light videos, in which high-resolution raw data is captured at video rate. At this level of darkness, the signal-to-noise ratio is extremely low (negative if measured in dB) and the traditional image processing pipeline generally breaks down. A new method is presented to address this challenging problem. By carefully designing a learning-based pipeline and introducing a new loss function to encourage temporal stability, we train a siamese network on static raw videos, for which ground truth is available, such that the network generalizes to videos of dynamic scenes at test time. Experimental results demonstrate that the presented approach outperforms state-of-the-art models for burst processing, per-frame processing, and blind temporal consistency.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c46503888ad5838ec044f9ca0cf8616d94e43c91

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation (2021, methodology): “MIT-Adobe FiveK dataset was originally collected for image global retouching; the SID [25] provides paired low/normal light raw data; a dataset of raw lowlight videos with the corresponding normal light videos captured at video rate was collected in [26].”
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, background): “This effort has expanded to include multi-image denoisers that can be applied to burst images or video frames [8, 46, 53].”
SNR-Aware Low-light Image Enhancement (2022, methodology): “Extensive experiments were conducted on 7 representative datasets: LOL (v1 [45], v2-real [53], & v2-synthetic [53]), SID [5], SMID [4], and SDSD (indoor & outdoor) [39].”
Benchmarking Low-Light Image Enhancement and Beyond (2021, background): “2018) 589 (sequences) Real Camera SMOID (Jiang and Zheng 2019) 202 (RAW videos) Real Camera DRV (Chen et al. 2019) 179 (RAW videos) Real Camera LOL (Wei et al.”
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement (2023, methodology): “We eveluate our method on LOL (v1 [54] and v2 [59]), SID [9], SMID [10], SDSD [48], and FiveK [5] datasets.”