Deep Colorization

By Zezhou Cheng, Qingxiong Yang, Bin Sheng. 2015

615 citations recorded by the source corpus.

This paper investigates into the colorization problem which converts a grayscale image to a colorful version. This is a very difficult problem and normally requires manual adjustment to achieve artifact-free quality. For instance, it normally requires human-labelled color scribbles on the grayscale target image or a careful selection of colorful reference images (e.g., capturing the same scene in the grayscale target image). Unlike the previous methods, this paper aims at a high-quality fully-automatic colorization method. With the assumption of a perfect patch matching technique, the use of an extremely large-scale reference database (that contains sufficient color images) is the most reliable solution to the colorization problem. However, patch matching noise will increase with respect to the size of the reference database in practice. Inspired by the recent success in deep learning techniques which provide amazing modeling of large-scale data, this paper re-formulates the colorization problem so that deep learning techniques can be directly employed. To ensure artifact-free quality, a joint bilateral filtering based post-processing step is proposed. Numerous experiments demonstrate that our method outperforms the state-of-art algorithms both in terms of quality and speed.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Colorful Image Colorization (2016, methodology): “Others have noted the easy availability of training data and trained convolutional neural networks (CNNs) to predict color on large datasets [1,2].”
Perceptual Losses for Real-Time Style Transfer and Super-Resolution (2016, methodology): “In [2] a feed-forward model is trained using a per-pixel loss to transform grayscale images to color.”
Learning Representations for Automatic Colorization (2016, methodology): “Some [5, 7] show promising results on typical scenes (e.”
Real-time user-guided image colorization with learned deep priors (2017, methodology): “Recently, fully automatic methods (Cheng et al. 2015; Deshpande et al. 2015; Iizuka et al. 2016; Isola et al. 2017; Larsson et al. 2016; Zhang et al. 2016) have been proposed.”
Let there be color! (2016, methodology): “We compare against the state of the art [Cheng et al. 2015] in Fig.”
Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception (2020, background): “The contemporary deep CNN-based generative models provide SOTA performance in learning such image-to-image translation for both terrestrial [14, 11] and underwater domains [35, 48].”
Lensless computational imaging through deep learning (2017, background): “…neural networks to “learn deeper;” and modern DNNs have shown spectacular success at solving “hard” computational problems, such as: playing complex games like Atari [17] and Go [18], object detection [19], and image restoration (e.g., colorization [20], deblurring [21–23], in-painting [24]).”
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks (2017, methodology): “The baseline approach for this problem is to predict new values for each pixel based on its local description that consists of various hand-crafted features [3].”
Fast Underwater Image Enhancement for Improved Visual Perception (2019, background): “The existing literature based on deep Convolutional Neural Networks (CNNs) provide good solutions for automatic image enhancement problems such as color/contrast adjustment [23, 8], colorization [42], dehazing [5], superresolution [10], etc.”
Perceptual Adversarial Networks for Image-to-Image Transformation (2017, background): “More image-to-image transformation tasks based on feed-forward CNNs, include, but not limited to, image colorization [10], depth estimations [33], [36], etc .”
ZM-Net: Real-time Zero-shot Image Manipulation Network (2017, methodology): “For example, in [1, 24], a CNN is trained to perform colorization on input images, and in [9, 22, 2, 12] to transform content images according to specific styles.”
Artificial intelligence in the creative industries: a review (2020, methodology): “techniques for colorization used a CNN with only three convolutional layers to convert a grayscale image into chrominance values and rened them with bilateral lters to generate a natural color image [82]. A deeper network, but still only with eight dilated convolutional layers, was proposed a year l”
Deep high dynamic range imaging of dynamic scenes (2017, background): “1 Inspired by the recent success of deep learning in a variety of applications such as colorization [Cheng et al. 2015; Iizuka et al. 2016] and view synthesis [Flynn et al.”
Colorization Transformer (2021, methodology): “The deterministic colorization techniques such as CIC (Zhang et al., 2016), LRAC (Larsson et al., 2016), LTBC (Iizuka et al., 2016), Pix2Pix (Isola et al., 2017) and DC (Cheng et al., 2015; Dahl, 2016) involve variations of CNNs to model per-pixel color information conditioned on the intensity.”