Multi-view stereo via volumetric graph-cuts

By George Vogiatzis, Philip H. S. Torr, Roberto Cipolla. 2005

372 citations recorded by the source corpus.

This paper presents a novel formulation for the multi-view scene reconstruction problem. While this formulation benefits from a volumetric scene representation, it is amenable to a computationally tractable global optimisation using Graph-cuts. The algorithm proposed uses the visual hull of the scene to infer occlusions and as a constraint on the topology of the scene. A photo consistency-based surface cost functional is defined and discretised with a weighted graph. The optimal surface under this discretised functional is obtained as the minimum cut solution of the weighted graph. Our method provides a viewpoint independent surface regularisation, approximate handling of occlusions and a tractable optimisation scheme. Promising experimental results on real scenes as well as a quantitative evaluation on a synthetic scene are presented.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning a Multi-View Stereo Machine (2017, neutral): “In the multi-view setting, early work focused on using silhouette information via visual hulls [32], incorporating photo-consistency to deal with concavities (photo hull) [29], and shape reﬁnement using optimization [55, 50, 7, 15].”
Discrete optimization of ray potentials for semantic 3D reconstruction (2015, neutral): “Regularizing the surface by penalizing the surface area has been proposed in the discrete graph cut formulation [16, 25] and also as continuous convex optimization formulation [28].”
Dynamic shape capture using multi-view photometric stereo (2009, methodology): “Recent approaches additionally employ global optimization to take into account smoothness constraints as well as the silhouettes of the object [Hernandez and Schmitt 2004; Vogiatzis et al. 2005; Furukawa and Ponce 2006; Hornung and Kobbelt 2006].”
Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency (2007, methodology): “Centre: the metric of [29] con tains falsely photo-consistent regions (e.”
Graph Cuts and Efficient N-D Image Segmentation (2006, methodology): “Combinatorial s/t graph cut framework for object segmentation was first proposed in Boykov and Jolly (2001) and recently further developed in (Boykov and Kolmogorov, 2003; Blake et al., 2004; Rother et al., 2004; Li et al., 2004; Kirsanov and Gortler, 2004; Li et al., 2005; Wang et al., 2005; Kolmog”
Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraint (2016, background): “ Zhu et al.[39] 0.4 / 99.2 0.45 / 95.7 0.38 / 98.3 0.48 / 95.4 Li et al.[20] 0.73 / 98.2 0.66 / 97.3 0.28 / 100 0.3 / 100 Wei et al. [34] 0.34 / 99.4 0.42 / 98.1 Xue et al. 0.3 / 99.1 Furukawa et al. [7] 0.49 / 99.6 0.47 / 99.6 0.63 / 99.3 0.33 / 99.8 0.28 / 99.8 0.37 / 99.2 Figure 5: Middlebury Mul”
Towards Internet-scale multi-view stereo (2010, methodology): “Many of the best performing MVS algorithms instead reconstruct a global 3D model directly from the input images [9, 13, 20, 24].”
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms (2006, methodology): “A number of methods define a volumetric MRF and use max-flow [6, 19, 47, 48] or multi-way graph cut [35] to extract an optimal surface.”
Reconstructing building interiors from images (2009, methodology): “By contrast, F 1 has either been an inflating term [23]—a small constant penalizing empty voxels, to avoid the degenerate solution of no surface at all—or has been informed by visibility information derived from depth maps [13].”
A Surface-Growing Approach to Multi-View Stereo Reconstruction (2007, methodology): “[23] by adding silhouette constraints to avoid the surface shrinkage that volumetric graph cuts are affected with.”
Consistent Depth Maps Recovery from a Video Sequence (2009, methodology): “With this observation, in this paper, we introduce a novel data term that combines the photo-consistency and geometric coherence constraints in a statistical way.”
Continuous depth estimation for multi-view stereo (2009, methodology): “In MVS area, Kolev [14] proposed a continuous global optimization algorithm parallel to discrete volumetric graph cuts [28].”
High Accuracy and Visibility-Consistent Dense Multiview Stereo (2012, neutral): “This includes space carving [49], [37], [8], [68], level sets [14], [32],[46], and volumetric graph cuts [65], [6], [28], [43], [61] (though [50], [27] propose regular volumetric grid adaptive to photoconsistency measures to push the resolution limit further).”
Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo (2008, methodology): “The use of a volumetric graph-cut to extract the surface was proposed in [14] and extended in [6] to include the robust NCC photoconsistency.”
A Comparative Study of Energy Minimization Methods for Markov Random Fields (2006, methodology): “Binary MRF’s are also widely used in medical image segmentation [12], video segmentation and merging [20, 34], stereo matching using minimal surfaces [35– 37], and video segmentation using stereo disparity cues [38] As previously mentioned, for the natural Potts model smoothness costs, the global mi”
Minimizing the Reprojection Error in Surface Reconstruction from Images (2007, background): “This can be done once and for all [8, 14, 23] or iteratively, alternating the computation of the visibility with the optimization of the functional [5, 15, 21].”
Accurate, Dense, and Robust Multi-View Stereopsis (2007, background): “They can be roughly classified into four classes according to the underlying object models: Voxel-based approaches [3], [4], [5], [6], [7], [8], [9] require knowing a bounding box that contains the scene, and their accuracy is limited by the resolution of the voxel grid.”
Continuous Global Optimization in Multiview 3D Reconstruction (2007, methodology): “This model relies on a classical foreground/background subdivision of the 3D space based on silhouette cues (Vogiatzis et al., 2005; Hornung and Kobbelt, 2006).”