Articulation-Aware Canonical Surface Mapping

By Nilesh Kulkarni, Abhinav Kumar Gupta, David F. Fouhey, Shubham Tulsiani. 2020

110 citations recorded by the source corpus.

We tackle the tasks of: 1) predicting a Canonical Surface Mapping (CSM) that indicates the mapping from 2D pixels to corresponding points on a canonical template shape , and 2) inferring the articulation and pose of the template corresponding to the input image. While previous approaches rely on keypoint supervision for learning, we present an approach that can learn without such annotations. Our key insight is that these tasks are geometrically related, and we can obtain supervisory signal via enforcing consistency among the predictions. We present results across a diverse set of animal object categories, showing that our method can learn articulation and CSM prediction from image collections using only foreground mask labels for training. We empirically show that allowing articulation helps learn more accurate CSM prediction, and that enforcing the consistency with predicted CSM is similarly critical for learning meaningful articulation.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Multiview Compressive Coding for 3D Reconstruction (2023, methodology): “Reminiscent of generalized cylinders [41], some introduce object-specific priors via category-specific 3D templates [26,30,32], pose [43] or symmetries [73].”
LRM: Large Reconstruction Model for Single Image to 3D (2023, unknown): “Leveraging 3D templates (Roth et al., 2016; Goel et al., 2020; Kanazawa et al., 2018; Kulkarni et al., 2020), semantics (Li et al., 2020), and poses (Bogo et al., 2016; Novotny et al., 2019) as shape priors have also been widely studied in category-specific reconstruction.”
Unsupervised Learning of 3D Object Categories from Videos in the Wild (2021, background): “Instead of assuming knowledge of pose, [28, 27, 14] assume a deformable 3D template.”
GAN-Supervised Dense Visual Alignment (2021, methodology): “Numbers for the 3D methods are reported from [45].”
Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction (2021, background): “[35, 34] map pixels to object-specific derformable template shapes.”
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild (2019, unknown): “3DV, VP ShapeNet, others [28]∗ I, 2DS, 3DTM Fit 3DTM to I Animals [29]∗ I, 2DS, 3DTM Pred.”