Continuous Markov Random Fields for Robust Stereo Estimation

By Koichiro Yamaguchi, Tamir Hazan, David A. McAllester, Raquel Urtasun. 2012

134 citations recorded by the source corpus.

In this paper we present a novel slanted-plane model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variables. This allows us to define potentials encoding the ownership of the pixels that compose the boundary between segments, as well as potentials encoding which junctions are physically possible. Our approach outperforms the state-of-the-art on Middlebury high resolution imagery [1] as well as in the more challenging KITTI dataset [2], while being more efficient than existing slanted plane MRF methods, taking on average 2 minutes to perform inference on high resolution imagery.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network (2014, background): “Machine learning techniques have also been applied in the stereo case, often obtaining better results while relaxing the need for careful camera alignment [8, 13, 21, 19].”
Deep Stereo: Learning to Predict New Views from the World's Imagery (2015, methodology): “Learning has been used in several ways, including estimating the parameters of more traditional models such as MRFs [36] and learning low-level correlation ﬁlters for disparity estimation [24, 17].”
Discrete-Continuous Depth Estimation from a Single Image (2014, methodology): “While this has achieved some success, as suggested in [31] in the context of stereo, the gradient-aware smoothing strategy often poorly reflects the real 3D scene observed in the image.”
Computing the stereo matching cost with a convolutional neural network (2014, unknown): “[18] 4.”
Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation (2014, methodology): “Unfortunately, these slanted plane methods have involved time-consuming optimization algorithms (several minutes per frame) such as particle belief propagation [30,31] or algorithms based on plane proposals with fusion moves and iterated cut-based segmentations [26].”
A multi-block-matching approach for stereo (2015, methodology): “It shows the top two ranking methods at that time: CVPR 1186 [23] and MC-CNN [23] both being anonymous at that time; plus some selected methods: Particle Convex Belief Propagation PCBP [24], SceneFlow [25], SGM with road scene adaption rdSGM [23], Weighted SGM wSGM [26] and summed normalized cross-c”
3D Scene Flow Estimation with a Piecewise Rigid Scene Model (2015, background): “While progress in dense binocular stereo (Bleyer et al. 2011b; Hirschmüller 2008; Yamaguchi et al. 2012, etc.) and optical ﬂow (Brox et al. 2004; Sun et al. 2010; Unger et al. 2012, among others) has been both steady and signiﬁcant over the years, the performance of 3D scene ﬂow algorithms (e.g.,…”
Piecewise Rigid Scene Flow (2013, background): “[26], who additionally penalize deviations from the (not segment-based) initialization.”
Blending Learning and Inference in Conditional Random Fields (2016, methodology): “The log-beliefs objective is concave and consists of two types of variables that are related to the learning and inference tasks respectively.”
Image Segmentation by Cascaded Region Agglomeration (2013, background): “, little or no “leakage” of true regions across oversegmentation boundaries), this speeds up reasoning, with little loss of accuracy for the downstream task, such as category-level segmentation [3, 7] or depth estimation [25].”
Semi-global Stereo Matching with Surface Orientation Priors (2017, methodology): “A more recent trend is to formulate stereo matching using continuous MRF frameworks [48].”
Efficient Deep Learning for Stereo Matching (2016, background): “The KITTI 2012 dataset contains 194 training and 195 test images.”
Unfolding an Indoor Origami World (2014, methodology): “These have been applied in the context of stereo [37], but have largely been ignored in the single-image layout community, apart from work on shape recovery using hand-marked folds [19].”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “Superpixels: An alternative way of modeling piecewise planarity is to explicitly partition the image into superpixels and modeling the surface at each superpixel as a slanted plane (Yamaguchi et al. (2012); Güney & Geiger (2015)).”
Displets: Resolving stereo ambiguities using object knowledge (2015, methodology): “Similarly to [64], we observe diminishing returns beyond 500 superpixels and chose 1000 superpixels as a reasonable tradeoff between accuracy and performance for all other experiments.”
Are we ready for autonomous driving? The KITTI vision benchmark suite (2012, unknown): “Stereo Non-Occluded All Density PCBP [46] 4.”
Patches, Planes and Probabilities: A Non-Local Prior for Volumetric 3D Reconstruction (2016, methodology): “Inspired by the success of non-local segmentation [25, 29] and stereo matching [19,32,53] techniques, we encourage planarity within coherent image segments in all viewpoints.”