Joint bilateral upsampling

By Johannes Peter Kopf, Michael F. Cohen, Dani Lischinski, M. Uyttendaele. ACM Transactions on Graphics, 2007

1,341 citations recorded by the source corpus.

Image analysis and enhancement tasks such as tone mapping, colorization, stereo depth, and photomontage, often require computing a solution (e.g., for exposure, chromaticity, disparity, labels) over the pixel grid. Computational and memory costs often require that a smaller solution be run over a downsampled image. Although general purpose upsampling methods can be used to interpolate the low resolution solution to the full resolution, these methods generally assume a smoothness prior for the interpolation. We demonstrate that in cases, such as those above, the available high resolution input image may be leveraged as a prior in the context of a joint bilateral upsampling procedure to produce a better high resolution solution. We show results for each of the applications above and compare them to traditional upsampling methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1798058243aad828b92b2f7fa50dd5237c8a37d8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Rolling Guidance Filter (2014, neutral): “A variant is joint/cross filtering [21,16,13,11].”
Depth Map Super-Resolution by Deep Multi-Scale Guidance (2016, methodology): “used the joint bilateral filter [1] to weight the degree of smoothing in each depth patch by considering the color similarity between the center pixel and its neighborhood [13].”
Guided Image Filtering (2010, methodology): “To reduce the running time, the chrominance channels are solved at a coarse resolution and upsampled under the guidance of the full resolution luminance image by the joint bilateral filter [ 21 ].”
Fast Global Image Smoothing Based on Weighted Least Squares (2014, methodology): “Thus, through this adaptive averaging process, our fast global smoother handles such noise-like structures better than the L0 smoothing, yet more efficiently.”
Deep bilateral learning for real-time image enhancement (2017, methodology): “Joint bilateral upsampling [Kopf et al. 2007] does this by using a bilateral filter on a high-resolution guidance map to produce a piecewise-smooth edgeaware upsampling.”
ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data (2021, methodology): “Initial classical approaches to color-guided depth upsampling include both optimization and filtering-based methods [38, 39].”
Pixel-Adaptive Convolutional Neural Networks (2019, neutral): “03 JBU [24] 4.”
Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model (2014, methodology): “Five methods applicable for hole ﬁlling are compared: Bicubic, IMLS [36], joint bilateral ﬁl-tering (JBF) [12], Guided [13], and CLMF0 [32].”
Deep Joint Image Filtering (2016, methodology): “Among them, JBU [4], GF [6] and Ham [13] are generic methods for joint image upsampling while MRF [16], TGV [3] and Park [2] are algorithms specifically designed for image guided depth upsampling.”
Sparsity Invariant CNNs (2017, background): “A popular choice for guided upsampling is guided bilateral ﬁltering [2, 7, 34, 42, 68].”
Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation (2016, methodology): “It can be seen that the RGB images reproduced by both the proposed method and the CNFM method are more consistent with the captured RGB images than that of [21], verifying that the proposed method and CNMF method can reconstruct more accurate hyperspectral images than the method of [21], which uses ”
Sparse Dictionary Learning for Edit Propagation of High-Resolution Images (2014, background): “Also related are a large body of works on edge-aware or edge-centric image processing [5, 11] and image upsampling techniques [12].”
Deepstrip: High-Resolution Boundary Refinement (2020, methodology): “For upsampling based approaches, we compare our approach with Bilinear Upsampling , Bilateral Solver [6] , Joint Bilateral Upsampling [20] ( JBU ), Guided Filtering [17] and Deep GF [40] .”
High quality depth map upsampling for 3D-TOF cameras (2011, methodology): “Yang et al. [19] used joint bilateral ﬁltering [8, 12] to interpolate the high-resolution depth values.”
Shading-based refinement on volumetric signed distance functions (2015, background): “…and depth edges, which can be exploited in a joint edge-preserving upsampling ﬁlter, such as a bilateral or multi-lateral ﬁlter [Lind-ner et al. 2007; Kopf et al. 2007; Chan et al. 2008; Dolson et al. 2010; Richardt et al. 2012], or explicitly phrased in an optimization problem [Park et al. 2011;…”
Light Field Image Processing: An Overview (2017, methodology): “To efficiently handle the large scale of light fields, they proposed to operate on a downsampled light field, and then upsample the results following [213].”
Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation (2013, methodology): “We compare standard interpolation methods as well as joint bilateral filtering [11] and image guided filtering”