Depth from Combining Defocus and Correspondence Using Light-Field Cameras

By Michael W. Tao, Sunil Hadap, Jitendra Malik, Ravi Ramamoorthi. IEEE International Conference on Computer Vision, 2013

554 citations recorded by the source corpus.

Light-field cameras have recently become available to the consumer market. An array of micro-lenses captures enough information that one can refocus images after acquisition, as well as shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. Thus, depth cues from both defocus and correspondence are available simultaneously in a single capture. Previously, defocus could be achieved only through multiple image exposures focused at different depths, while correspondence cues needed multiple exposures at different viewpoints or multiple cameras, moreover, both cues could not easily be obtained together. In this paper, we present a novel simple and principled algorithm that computes dense depth estimation by combining both defocus and correspondence depth cues. We analyze the x-u 2D epipolar image (EPI), where by convention we assume the spatial x coordinate is horizontal and the angular u coordinate is vertical (our final algorithm uses the full 4D EPI). We show that defocus depth cues are obtained by computing the horizontal (spatial) variance after vertical (angular) integration, and correspondence depth cues by computing the vertical (angular) variance. We then show how to combine the two cues into a high quality depth map, suitable for computer vision applications such as matting, full control of depth-of-field, and surface reconstruction.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/ae818782111e26d42cdbb3d850b1f17b9578629a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Soft 3D reconstruction for view synthesis (2017, result): “Since [Kalantari et al. 2016] compares favorably to [Zhang et al. 2016] and [Wanner and Goldluecke 2014] (using depth estimation of [Tao et al. 2013], [Wang et al. 2015] and [Jeon et al. 2015]) we refer to the reader to [Kalantari et al. 2016] for further comparisons with these methods.”
A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields (2016, background): “For this reason, recent papers often resort to showing qualitative results on real-world datasets to showcase their improved results [2,6,7], but performance is very difficult to judge without ground truth.”
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection (2019, background): “Our dataset contains 800 indoor and 400 outdoor scenes paired with corresponding depth maps [40] and ground truths.”
Learning a Deep Convolutional Network for Light-Field Image Super-Resolution (2015, background): “With richer angular information in a LF image, many studies have shown the potential to improve the performance of many computer vision applications such as alpha matting [6], saliency detection [17], LF panorama [3], and depth reconstruction [27, 33, 13].”
Learning-based view synthesis for light field cameras (2016, unknown): “Since our disparity estimation CNN is trained to directly minimize the synthesis error, our estimated disparities are more suitable for this application than existing disparity estimation techniques [Wanner and Goldluecke 2012; Tao et al. 2013; Wang et al. 2015; Jeon et al. 2015] (see Fig.”
Robust depth estimation for light field via spinning parallelogram operator (2016, unknown): “[18] (CDC), ho used both stereo and defocus cues to better approximate the rue depth maps of Lytro images.”
Light Field Image Processing: An Overview (2017, background): “MRF are widely used for problems that involve feature point matching [96]–[99], [102].”