Colorization using optimization

By Anat Levin, Dani Lischinski, Yair Weiss. 2004

1,823 citations recorded by the source corpus.

Colorization is a computer-assisted process of adding color to a monochrome image or movie. The process typically involves segmenting images into regions and tracking these regions across image sequences. Neither of these tasks can be performed reliably in practice; consequently, colorization requires considerable user intervention and remains a tedious, time-consuming, and expensive task.In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise; neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Visual Manipulation on the Natural Image Manifold (2016, background): “Examples of basic editing include changing the color properties of an image either globally [6] or locally [7].”
Learning Representations for Automatic Colorization (2016, methodology): “However, most methods still require some level of user input [3, 6, 13, 19, 21, 33].”
Indoor Semantic Segmentation using depth information (2013, methodology): “The dataset is provided with the original raw depth data that contain missing values, with code using [7] to inpaint the depth images.”
Scribbler: Controlling Deep Image Synthesis with Sketch and Color (2016, background): “Controllable Colorization Our color control strokes are inspired by Colorization Using Optimization [25] which interpolates sparse color strokes such that color changes tend to happen at intensity boundaries.”
Learning Affinity via Spatial Propagation Networks (2017, methodology): “Numerous methods explicitly design affinity matrices for image filtering [29, 10], colorization [15], matting [16] and image segmentation [14] based on the characterstics of the problem.”
Real-time user-guided image colorization with learned deep priors (2017, methodology): “Prior interactive colorization work focused on local control, such as user strokes (Huang et al. 2005; Levin et al. 2004).”
Exemplar-Based Face Parsing (2013, background): “Face Image Reconstruction and Synthesis Examplarbased face image reconstruction/synthesis is applicable for various face image editing tasks, including grayscale image colorization [10] and automatic face image retouching [5].”
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition (2008, methodology): “It wou ld be possible to improve the quality of the results by using a method such as Levin et al. [ 24 ] which would take color cues from the sibling images and propagate within the input image in an edgeaware manner.”
Introduction to Semi-Supervised Learning (2009, background): “Some theoretical analysis of graph-based learning can be found in [91, 178, 195], and applications in [73, 78, 111, 102, 136, 138, 139].”
Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network (2016, methodology): “method BLF [18] WLS [20] RTV [11] WMF [36] EPLL [41] Levin [42] Xu et al.”
Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting (2009, background): “Our work is inspired by [33] [34], where the authors, following [16], show how to use distance functions for image colorization.”
RGBD Salient Object Detection: A Benchmark and Algorithms (2014, methodology): “To obtain a filled and smoothed depth image, we adopt a colorization scheme [26] to repair the calibrated depth map.”
Self-similarity-based image colorization (2014, methodology): “By customizing Levin et al.'s work [1], we formulate the colorization problem as a quadratic programming problem.”
Colorization by example (2005, methodology): “For example, Levin et al. [LLW04] recently proposed a simple yet effective user-guided colorization method.”
Manga colorization (2006, methodology): “Existing colorization methods [Levin et al. 2004; Sykora et al. 2004] mainly rely on a " rough " continuity of gray levels to grow the affective regions, so as to segment the image into color regions.”
The Fast Bilateral Solver (2015, methodology): “[23] presented an Method Err Time (sec) ∗Kiechle et al.”
Learning Large-Scale Automatic Image Colorization (2015, neutral): “A good review appears in [12] .”
Indoor Segmentation and Support Inference from RGBD Images (2012, methodology): “We are provided with registered RGB and depth images, with in-painted depth pixels [15].”