A Closed-Form Solution to Non-Rigid Shape and Motion Recovery

By Jing Xiao, Jinxiang Chai, Takeo Kanade. International Journal of Computer Vision, 2004

375 citations recorded by the source corpus.

Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem (Bregler, C., Hertzmann, A., and Biermann, H. 2000. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Brand, M. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition; Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. In Proc. Int. Conf. Computer Vision and Pattern Recognition) utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique. An arbitrary linear transformation of the bases produces another set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/33ca9c265a112f09544e4d368b83e53cea094260

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Factorization for non-rigid and articulated structure using metric projections (2009, background): “We show successful results of our algorithms on synthetic and real sequences of both deformable and articulated data.”
Simultaneous Localization, Mapping and Moving Object Tracking (2007, methodology): “The ideas based on the factorization techniques and the shape basis representation are presented in Bregler et al. (2000), Brand (2001), Torresani et al. (2001) and Xiao et al. (2004), where different constraints are used for finding the solution.”
2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting (2007, methodology): “Equation (29) assumes that the appearance vectors Ai (u) have been orthonormalized.”
A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate (2006, background): “– The trajectories of a non-rigid object can be approximated by different weighings of a number of key shapes ([3][4][5]) and, as shown below, lie in a linear subspace of dimension no more than 3k + 1.”
Model evolution: An incremental approach to non-rigid structure from motion (2010, background): “After the pioneering work by Bregler et al. [5], a number of excellent solutions [3, 12, 10, 9, 4, 14 , 15, 11, 13, 2] have been proposed to solve NRSFM in the past decade.”
Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors (2008, background): “The main appeal of these approaches is that they decompose the problem into much simpler ones.”
Visual SLAM and Structure from Motion in Dynamic Environments (2018, background): “[176] introduce basis constraints such that the nonrigid factorization problem can be solved in a closed-form solution.”
A Simple Prior-Free Method for Non-rigid Structure-from-Motion Factorization (2012, methodology): “One of the primary causes to such difficulty is due to the inherent basis ambiguity of the nonrigid problem ([26]).”
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (2019, background): “A challenge with NR-SFM is the ambiguity in decomposing variations in the 3D shape of an object into viewpoint changes (rigid motions) and internal object deformations [40].”
Real-time combined 2D+3D active appearance models (2004, methodology): “We used a linear non-rigid structure-from-motion algorithm[12] that does not suffer from the local minima that other non-linear algorithms do.”