Fusion of time-of-flight depth and stereo for high accuracy depth maps

By Jiejie Zhu, Liang Wang, Ruigang Yang, James Davis. 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008

320 citations recorded by the source corpus.

Time-of-flight range sensors have error characteristics which are complementary to passive stereo. They provide real time depth estimates in conditions where passive stereo does not work well, such as on white walls. In contrast, these sensors are noisy and often perform poorly on the textured scenes for which stereo excels. We introduce a method for combining the results from both methods that performs better than either alone. A depth probability distribution function from each method is calculated and then merged. In addition, stereo methods have long used global methods such as belief propagation and graph cuts to improve results, and we apply these methods to this sensor. Since time-of-flight devices have primarily been used as individual sensors, they are typically poorly calibrated. We introduce a method that substantially improves upon the manufacturerpsilas calibration. We show that these techniques lead to improved accuracy and robustness.

Subfields: Computer Vision and Pattern Recognition, Instrumentation

View Paper: https://www.semanticscholar.org/paper/ac084ef7da5e035160cf1194e68943ccdcfaa0f6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Instrumentation

Citations in the corpus

High-accuracy 3D sensing for mobile manipulation: Improving object detection and door opening (2009, background): “The mobile manipulator enters several offices and searches for an object class, recording the detected locations.”
Unsupervised Intrinsic Calibration of Depth Sensors via SLAM (2013, background): “Time-offlight (ToF) sensor intrinsic calibration using calibration targets has been well-studied on its own [15, 16, 7] and in the context of sensor fusion [21, 11].”
Joint Depth and Color Camera Calibration with Distortion Correction (2012, background): “Most structured light systems can calibrate the projector and camera separately.”
Lock-in Time-of-Flight (ToF) Cameras: A Survey (2011, background): “This has been applied for a small depth range (only 400 mm) [65].”
3D shape scanning with a time-of-flight camera (2010, background): “Others tried to attack ToF camera deficiencies by combining them with normal color cameras, e.g. [26] [2].”
Real-time shading-based refinement for consumer depth cameras (2014, background): “Similar ideas have been explored for joint reconstruction using stereo images and depth data, where photometric constraints from stereo can be exploited for further data refinement [Beder et al. 2007; Zhu et al. 2008; Gudmundsson et al. 2008].”