Deep photo: model-based photograph enhancement and viewing

By Johannes Peter Kopf, Boris Neubert, Billy Chen, Michael F. Cohen, Daniel Cohen-Or, Oliver Deussen, M. Uyttendaele, Dani Lischinski. ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, 2008

846 citations recorded by the source corpus.

In this paper, we introduce a novel system for browsing, enhancing, and manipulating casual outdoor photographs by combining them with already existing georeferenced digital terrain and urban models. A simple interactive registration process is used to align a photograph with such a model. Once the photograph and the model have been registered, an abundance of information, such as depth, texture, and GIS data, becomes immediately available to our system. This information, in turn, enables a variety of operations, ranging from dehazing and relighting the photograph, to novel view synthesis, and overlaying with geographic information. We describe the implementation of a number of these applications and discuss possible extensions. Our results show that augmenting photographs with already available 3D models of the world supports a wide variety of new ways for us to experience and interact with our everyday snapshots.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/85ed33334c6c3bf7e75d5c3f12e07abd4aa50fa9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images (2020, background): “Image dehazing is an ill-posed problem that has been solved initially based on additional information such as atmospheric cues [18, 32], multiple images captured with polarization filters [33, 37], or known depth information [25, 40].”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, background): “In recent years, data-driven CNN architectures [7, 18, 92, 93, 105, 107] have been shown to outperform conventional restoration approaches [26,36,53,75].”
Initial results in underwater single image dehazing (2010, background): “In order to resolve the ambiguity between the depth and the direct transmission, many existing dehazing techniques rely on additional hardware such as polarization filters [3, 4], additional information such as the depth of the scene [5], or multiple images captured under different conditions [2, 6]”
Densely Connected Pyramid Dehazing Network (2018, methodology): “To overcome the degradations caused by haze, image and video-based haze removal algorithms have been proposed in the literature [33, 5, 42, 3, 13, 21, 27, 51, 24, 57, 8, 10, 9, 34].”
Referenceless Prediction of Perceptual Fog Density and Perceptual Image Defogging (2015, methodology): “Next, we compared the defogged images obtained using the models of Tan [10], Fattal [11], Kopf [8], He et al.”
DehazeNet: An End-to-End System for Single Image Haze Removal (2016, methodology): “Depth-based methods [6] require some depth information from user inputs or known 3D models.”
D-HAZY: A dataset to evaluate quantitatively dehazing algorithms (2016, background): “While earlier dehazing approaches employ additional information such as multiple images [3] or a rough estimate of the depth [4], recent techniques have tackled this problem by using only the information of a single hazy input image [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16].”
Efficient Image Dehazing with Boundary Constraint and Contextual Regularization (2013, background): “[6] propose to dehaze an image by using the scene depth information directly accessible in the georeferenced digital terrain or city models.”
Dehazing Using Color-Lines (2014, methodology): “The resulting images are meant to mimic the way natural images images are modeled by Lee et al. [2001]. We used this procedure twice and in the second image we generated (the DC Squares) we made sure that, when sampling the reflectance values, at least one channel is set to zero in order to meet the”
Weighted Guided Image Filtering (2016, methodology): “(a, h) input images; (b, i) de-hazed images by the algorithm in [34]; (c, j) de-hazed images by the algorithm in [27]; (d, k) de-hazed images by the algorithm in [33]; (e, l) de-hazed images by the algorithm in [7]; (f, m) de-hazed images by the algorithm in [14]; and (g, n) de-hazed images by the p”
GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing (2019, background): “Early works on image dehazing either require multiple images of the same scene taken under different conditions [30, 32, 20, 22, 24] or side information acquired from other sources [23, 12].”
Optimized contrast enhancement for real-time image and video dehazing (2013, background): “Also, Kopf et al. [6] employed the prior knowledge of the scene geometry for dehazing.”
Single Image Dehazing by Multi-Scale Fusion (2013, methodology): “This feature is achieved also by the methods of Fattal [6], He et al. [8] and Kopf et al. [4].”
Single image haze removal using dark channel prior (2009, methodology): “We also compare our method with Kopf et al .’s very recent work [5] in Figure 11.”
Bayesian Defogging (2012, methodology): “10 Factorization results of real-world foggy images, of (a) a view of Manhattan, and (b), (c) two scenic views of the Half Dome, compared with those of another method by Kopf et al. (2008b). For each example the input image, the albedo image by Kopf et al.”
Worldwide Pose Estimation Using 3D Point Clouds (2012, background): “Localizing precisely where a photo or video was taken is a key problem in computer vision with a broad range of applications, including consumer photography (“where did I take these photos again?”), augmented reality [1], photo editing [2], and autonomous navigation [3].”
Single Image Dehazing via Multi-scale Convolutional Neural Networks with Holistic Edges (2019, methodology): “In Kopf et al. (2008) an approximated 3D geometrical model of the scene is assumed to be available, from which a data-driven dehazing method is developed for image dehazing.”
I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images (2018, background): “Oldest approaches rely on atmospheric cues [1], [2], multiple images captured with polarization ﬁlters [3], [4], or known depth [5], [6].”
Color Balance and Fusion for Underwater Image Enhancement (2018, background): “Deep Photo system [17] is able to restore images”