Monocular Model-Based 3D Tracking of Rigid Objects: A Survey

By Vincent Lepetit, Pascal Fua. Foundations and Trends in Computer Graphics and Vision, 2005

768 citations recorded by the source corpus.

Many applications require tracking of complex 3D objects. These include visual servoing of robotic arms on specific target objects, Augmented Reality systems that require real-time registration of the object to be augmented, and head tracking systems that sophisticated interfaces can use. Computer Vision offers solutions that are cheap, practical and non-invasive.This survey reviews the different techniques and approaches that have been developed by industry and research. First, important mathematical tools are introduced: Camera representation, robust estimation and uncertainty estimation. Then a comprehensive study is given of the numerous approaches developed by the Augmented Reality and Robotics communities, beginning with those that are based on point or planar fiducial marks and moving on to those that avoid the need to engineer the environment by relying on natural features such as edges, texture or interest. Recent advances that avoid manual initialization and failures due to fast motion are also presented. The survery concludes with the different possible choices that should be made when implementing a 3D tracking system and a discussion of the future of vision-based 3D tracking.Because it encompasses many computer vision techniques from low-level vision to 3D geometry and includes a comprehensive study of the massive literature on the subject, this survey should be the handbook of the student, the researcher, or the engineer who wants to implement a 3D tracking system.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/4c51ca3b2bd924d689713862d0fdbaafa5b24e5b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Geology

Citations in the corpus

The MOPED framework: Object recognition and pose estimation for manipulation (2011, methodology): “In this section, we formalize these inputs and outputs and introduce the terminology we use throughout the paper.”
MonoSLAM: Real-Time Single Camera SLAM (2007, background): “about the structure of the observed scene, either via the placement of fiducial targets or a prior map-learning stage (see [51] for a review).”
Towards Fast, Accurate and Stable 3D Dense Face Alignment (2020, background): “However, the three Euler angles will cause the gimbal lock [31] when faces are close to the profile view.”
Multiple 3D Object tracking for augmented reality (2008, background): “Single 3D object tracking using natural features has now been well explored by the Computer Vision and Augmented Reality communities [4].”
Segmentation-Driven 6D Object Pose Estimation (2018, methodology): “Traditionally, it has been handled by establishing correspondences between the object’s known 3D model and 2D pixel locations, followed by using the Perspective-n-Point (PnP) algorithm to compute the 6 pose parameters [19, 38, 43].”
Face Alignment in Full Pose Range: A 3D Total Solution (2015, background): “four dimensional unit quaternion [56] 1⁄2q0; q1; q2; q3 instead of the Euler angles to formulate the rotation.”
Visual simultaneous localization and mapping: a survey (2012, background): “Yilmaz et al. (2006),Cannons (2008) and Lepetit and Fua (2005) present a study of the state-of-the-art of techniques to perform tracking based on features, contours or regions.”
Real-Time Seamless Single Shot 6D Object Pose Prediction (2017, background): “However, they only reliably handle textured objects in high resolution images [15].”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, methodology): “However, traditional methods have difficulty in handling texture-less objects and processing low-resolution images [20].”
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN Supervision (2016, methodology): “Our solution differs from Perspective-n-Point 6DOF rigid pose estimation [39], structure-from-motion, and from the convex approach of Zhou et al.”