Learning Image-Adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-Time

By Huiyu Zeng, Jianrui Cai, Lida Li, Zisheng Cao, Lei Zhang. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

363 citations recorded by the source corpus.

Recent years have witnessed the increasing popularity of learning based methods to enhance the color and tone of photos. However, many existing photo enhancement methods either deliver unsatisfactory results or consume too much computational and memory resources, hindering their application to high-resolution images (usually with more than 12 megapixels) in practice. In this paper, we learn image-adaptive 3-dimensional lookup tables (3D LUTs) to achieve fast and robust photo enhancement. 3D LUTs are widely used for manipulating color and tone of photos, but they are usually manually tuned and fixed in camera imaging pipeline or photo editing tools. We, for the first time to our best knowledge, propose to learn 3D LUTs from annotated data using pairwise or unpaired learning. More importantly, our learned 3D LUT is image-adaptive for flexible photo enhancement. We learn multiple basis 3D LUTs and a small convolutional neural network (CNN) simultaneously in an end-to-end manner. The small CNN works on the down-sampled version of the input image to predict content-dependent weights to fuse the multiple basis 3D LUTs into an image-adaptive one, which is employed to transform the color and tone of source images efficiently. Our model contains less than 600K parameters and takes less than 2 ms to process an image of 4K resolution using one Titan RTX GPU. While being highly efficient, our model also outperforms the state-of-the-art photo enhancement methods by a large margin in terms of PSNR, SSIM and a color difference metric on two publically available benchmark datasets. Code will be released at https://github.com/HuiZeng/Image-Adaptive-3DLUT.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c07efa053302fae65ee19aa053001a73cfa431e8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2021, background): “Moreover, although some general photo enhancement methods [45], [46], [47], [48], [49], [50], [51], [52], [53] can improve the brightness of images to some extent, we omit them in this survey as they are not designed to handle diverse low-light conditions.”
SNR-Aware Low-light Image Enhancement (2022, methodology): “612 Methods LPNet [26] MIR-Net [59] RF [24] 3DLUT [60] A3DLUT [42] Band [52] EG [20] Retinex [29] Sparse [53] IPT [6] Uformer [44] Ours PSNR 19.”
Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method (2022, methodology): “Data-driven methods (Wei et al. 2018; Zhang et al. 2020; Lim and Kim 2020; Ni et al. 2020; Zeng et al. 2020; Wang et al. 2022a, 2021; Yang et al. 2022) have been successfully applied to the LLIE task.”
Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions (2021, neutral): “To achieve adaptive image enhancement, (Hu et al. 2018; Yu et al. 2018; Zeng et al. 2020) employed a small CNN to flexibly learn the hyperparameters of image transformation.”