Joint 3d Estimation of Vehicles and Scene Flow

By Moritz Menze, Christian Heipke, Andreas Geiger. 2015

380 citations recorded by the source corpus.

Abstract. driving. While much progress has been made in recent years, imaging conditions in natural outdoor environments are still very challenging for current reconstruction and recognition methods. In this paper, we propose a novel unified approach which reasons jointly about 3D scene flow as well as the pose, shape and motion of vehicles in the scene. Towards this goal, we incorporate a deformable CAD model into a slanted-plane conditional random field for scene flow estimation and enforce shape consistency between the rendered 3D models and the parameters of all superpixels in the image. The association of superpixels to objects is established by an index variable which implicitly enables model selection. We evaluate our approach on the challenging KITTI scene flow dataset in terms of object and scene flow estimation. Our results provide a prove of concept and demonstrate the usefulness of our method.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, background): “Dataset MiddleBury[12] KITTI2012[13] KITTI2015[17] Sintel[14] Apollo [18] Scene Flow[15] IRS(ours) Synthetic(S) / Natural(N) N N N S N S S Scene Lab Road Road Outdoor Road Outdoor Indoor Resolution 2960x1942 1226x370 1242x375 1024x436 3130x960 960x540 960x540 Training/Testing Data Size 23/10 194/194”
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching (2020, methodology): “We consider the following datasets: SceneFlow [37], KITTI 2012 [15], KITTI 2015 [38], ETH3D [44], Middlebury dataset V3 [42].”
AutoFlow: Learning a Better Training Set for Optical Flow (2021, methodology): “KITTI [12] was created using LIDAR for static scenes and later extended to rigidly moving cars [33] for autonomous driving applications.”
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching (2021, methodology): “We evaluate RAFT-Stereo on ETH3D [29], Middlebury [28] and KITTI-2015 [24].”
Joint 3D Object and Layout Inference from a Single RGB-D Image (2015, methodology): “Furthermore, we plan to extend our model to object based understanding of dynamic scenes from RGB/RGB-D video sequences by reasoning about 3D scene flow [28,29].”
Learning to Estimate Hidden Motions with Global Motion Aggregation (2021, methodology): “We evaluate our approach on the Sintel dataset [8] and the KITTI 2015 optical flow dataset [28].”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “They further present an extension of this model in Menze et al. (2015b) where the pose and 3D shape of the objects are inferred in addition to the rigid motion and the segmentation.”
End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, methodology): “Following the KITTI 2015 benchmark [38], we report the percentage of pixels which have an end-pointerror larger than three pixels and 5% of the ground-truth flow, also done in [69].”
Consistent video depth estimation (2020, methodology): “2012] and KITTI2015 [Menze et al. 2015] training sets, FlowNet2ft-kitti, for extracting dense correspondence across frames.”
FlowNet3D: Learning Scene Flow in 3D Point Clouds (2018, methodology): “We use the KITTI scene flow dataset [17, 16], which is designed for evaluations of RGB stereo based methods.”