On 3D scene flow and structure estimation

By Ye Zhang, Chandra Kambhamettu. 2001

93 citations recorded by the source corpus.

In this paper, novel algorithms computing dense 3D scene flow from multiview image sequences are described. A new hierarchical rule-based stereo matching algorithm is presented to estimate the initial disparity map. Different available constraints under a multiview camera setup are investigated and then utilized in the proposed motion estimation algorithms. We show two different formulations for 3D scene flow computation. One formulation assumes that initial disparity map is accurate while the other does not make this assumption. Image segmentation information is used to maintain the motion and depth discontinuities. Iterative implementations are used to successfully compute 3D scene flow and structure at every point in the reference image. Novel hard constraints are introduced in this paper to make the algorithms more accurate and robust. Promising experimental results are seen by applying our algorithms to real imagery.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “Since this work, several methods [64, 57, 44] have been proposed based on stereo or multiple view camera systems.”
Multi-view Scene Flow Estimation: A View Centered Variational Approach (2010, background): “Decoupling the spatio-temporal information leads to sequential estimation of scene flow and structure (e.g., [17, 18, 22, 23, 3, 12, 20]).”
Efficient Dense Scene Flow from Sparse or Dense Stereo Data (2008, background): “The succeeding works in [7,10,19] presented energy minimisation frameworks including regularisation constraints to provide dense scene flow.”
Stereoscopic Scene Flow Computation for 3D Motion Understanding (2011, methodology): “The succeeding works in Huguet and Devernay (2007), Min and Sohn (2006), Zhang and Kambhamettu (2001) presented energy minimisation frameworks including regularisation constraints to provide dense scene flow.”
Modelling dynamic scenes by registering multi-view image sequences (2005, methodology): “Similarly, some scene flow methods [30, 3, 15] use the spatio-temporal derivatives of the input images.”
Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score (2007, methodology): “This assumption is also present in many methods for scene flow estimation, through the use of the spatio-temporal derivatives of the input images [39, 4, 21].”
A Variational Method for Scene Flow Estimation from Stereo Sequences (2007, background): “Zhang and Kambhamettu [22] achieve this by first segmenting the scene, and then applying piecewise regularization, but that problem can also be solved by using an appropriate regularization functional.”
Spacetime stereo: shape recovery for dynamic scenes (2003, background): “They later improve their results by fitting parametric motion to each local image region obtained by color segmentation, so that discontinuities are preserved [21].”
High performance imaging using large camera arrays (2005, methodology): “2004] and analysis of structure and motion in dynamic scenes [Tao et al. 2001; Zhang and Kambhamettu 2001].”
SphereFlow: 6 DoF Scene Flow from RGB-D Pairs (2014, methodology): “RGB scene flow methods include Huguet and Devernay [15], Min and Sohn [18], Zhang and Kambhamettu [32], Basha et al. [4], ˇ Cech et al. [25], and Vogel et al. [27].”
High-quality video view interpolation using a layered representation (2004, methodology): “Tao et al. [2001] used a planar constraint, while Zhang and Kambhamettu [2001] used the segments for local support.”
A primal-dual framework for real-time dense RGB-D scene flow (2015, methodology): “A global variational approach is presented in [5] where both the optical flow and depth flow are estimated simultaneously using quadratic regularization.”