Non-rigid dense correspondence with applications for image enhancement

By Yoav HaCohen, Eli Shechtman, Dan B. Goldman, Dani Lischinski. ACM Transactions on Graphics, 2011

384 citations recorded by the source corpus.

This paper presents a new efficient method for recovering reliable local sets of dense correspondences between two images with some shared content. Our method is designed for pairs of images depicting similar regions acquired by different cameras and lenses, under non-rigid transformations, under different lighting, and over different backgrounds. We utilize a new coarse-to-fine scheme in which nearest-neighbor field computations using Generalized PatchMatch [Barnes et al. 2010] are interleaved with fitting a global non-linear parametric color model and aggregating consistent matching regions using locally adaptive constraints. Compared to previous correspondence approaches, our method combines the best of two worlds: It is dense, like optical flow and stereo reconstruction methods, and it is also robust to geometric and photometric variations, like sparse feature matching. We demonstrate the usefulness of our method using three applications for automatic example-based photograph enhancement: adjusting the tonal characteristics of a source image to match a reference, transferring a known mask to a new image, and kernel estimation for image deblurring.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c802c5f223d294d58600f80f8c83517e9acb5e68

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Single image super-resolution from transformed self-exemplars (2015, methodology): “In this paper, we combine the explicit search strategy of [2] [17] [8], along with the perspective deformation estimation approach of [20].”
Light field video capture using a learning-based hybrid imaging system (2017, positive): “Since the DSLR view does not exactly match the central view of the Lytro camera in terms of viewpoint and color statistics, we first calibrate them by estimating a flow between the two views using non rigid dense correspondence (NRDC) (HaCohen et al. 2011).”
Convolutional Neural Network Architecture for Geometric Matching (2017, background): “E STIMATING correspondences between images is one of the fundamental problems in computer vision [1], [2] with applications ranging from large-scale 3-D reconstruction [3] to image manipulation [4] and semantic segmentation [5].”
Transformation guided image completion (2013, background): “, scale [16], similarity transform [19, 4, 10] and similarity with flip over y axis [7].”
Photo Sequencing (2012, methodology): “To overcome this challenge, we first find a dense correspondence between each image and the reference image, using the Non-Rigid Dense Correspondence (NRDC) algorithm HaCohen et al. (2011).”
GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence (2017, background): “More generally, our work is related to optical flow [13, 23, 4, 43, 33, 21, 1, 46], point based coherence techniques [48, 18, 29], patch-match based matchers [12] which directly use smoothness to help match estimation.”
Sequential line search for efficient visual design optimization by crowds (2017, methodology): “Note that our framework can also handle tonal curves by parame-terizing them ( e.g., [HaCohen et al. 2011]), though we did not use them in this example.”
Neural best-buddies (2018, background): “NRDC [HaCohen et al. 2011] extends generalized PatchMatch to cope with significant geometric and photometric variations of the same content.”
Robust patch-based hdr reconstruction of dynamic scenes (2012, background): “2010], and correspondence-based property transfer [HaCohen et al. 2011].”
Transfusive image manipulation (2012, methodology): “Another possibility would be to use the warps of previous works such as [Hasinoff et al. 2010; HaCohen et al. 2011] as initialization for ours.”
Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation (2015, background): “Promising is also to estimate patch deformations by random search [15].”
Deep high dynamic range imaging of dynamic scenes (2017, methodology): “Hu et al. [2012] align the images by finding dense correspondences using HaCohen et al.’s method [2011].”
Soft 3D reconstruction for view synthesis (2017, methodology): “We also align colors as a preprocess using a method similar to [HaCohen et al. 2011].”
Image melding (2012, methodology): “But in general, aligning photos under these variations is a challenging problem in itself [HaCohen et al. 2011].”