What Shape Are Dolphins? Building 3D Morphable Models from 2D Images

By Thomas Joseph Cashman, Andrew William Fitzgibbon. 2013

175 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Learning Category-Specific Mesh Reconstruction from Image Collections (2018, background): “Cashman and Fitzgibbon [3] use keypoint correspondences and segmentation masks to learn a morphable model of dolphins from images.”
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency (2017, background): “r-instance supervision. 2. Related Work Object Reconstruction from Image-based Annotations. Blanz and Vetter [1] demonstrated the use of a morphable model to capture 3D shapes. Cashman and Fitzgibbon [3] learned these models for complex categories like dolphins using object silhouettes and keypoint ”
Hierarchical Surface Prediction for 3D Object Reconstruction (2017, background): “Other works propose to learn a shape model from silhouettes [3] or silhouettes and keypoints [16].”
Lifting Object Detection Datasets into 3D (2015, positive): “To account for differences between the 3D exemplars and the object depicted, Su and Guibas [26] choose a few exemplars that appear to be similar to the depicted shape and combine them to produce a single depth map.”
Category-specific object reconstruction from a single image (2014, background): “Some interesting recent papers have extended variational approaches to handle categories [12, 13] but typically require some form of 3D annotations to bootstrap models.”
Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach (2015, background): “Recently, there was growing interest in reconstructing category-specific object models from a collection of single images of different instances with the same category [19], [20], [21], [22].”
Reconstructing PASCAL VOC (2014, methodology): “In order to partially overcome the need for 3D data, [8] proposes a hybrid method that uses a single 3D shape together with 2D information in order to build a morphable model for classes such as dolphins or pigeons.”
WarpNet: Weakly Supervised Matching for Single-View Reconstruction (2016, methodology): “Recently, methods have been proposed to match instances across categories, relying on supervision in the form of part (keypoint) annotations [7, 15, 29] or 3D CAD models [3, 8] to augment appearance information with shape priors.”
Interactive 3D Modeling with a Generative Adversarial Network (2017, methodology): “More recent work has utilized analysis of part-based templates [Cashman and Fitzgibbon 2013; Kraevoy et al. 2009], stability [Averkiou et al. 2014], functionality [Sageman-Furnas et al. 2015], ergonomics [Zheng et al. 2016], and other analyses to guide interactive manipulation.”
User-Specific Hand Modeling from Monocular Depth Sequences (2014, background): “Having built a PCM (or having simultaneously discovered alignment and PCM [10, 6]), a wide variety of impressive applications are enabled, such as the fitting of accurate 3D body models to a small number of range scans [30], or accurate hand or body tracking [7, 9].”
Nerfies: Deformable Neural Radiance Fields (2020, background): “Domain-Specific Modeling: Many reconstruction methods use domain-specific knowledge to model the shape and appearance of categories with limited topological variation, such as faces [5, 7, 9], human bodies [30, 58], and animals [12, 64].”
Balloon Shapes: Reconstructing and Deforming Objects with Volume from Images (2013, methodology): “In order to overcome this difficulty, the method in [7] shows how to build a morphable model from considerably less data which can be easily obtained.”
Unsupervised Learning of 3D Object Categories from Videos in the Wild (2021, methodology): “Early reconstruction methods for 3D object categories used Non-Rigid SfM (NR-SfM) applied to 2D keypoint annotations [4, 58, 3].”
Pose Induction for Novel Object Categories (2015, background): “Learning based approaches which also allow shape deformations [2] provide an alternative solution but typically rely on some 3D initialization [6].”
Review of statistical shape spaces for 3D data with comparative analysis for human faces (2012, background): “While recently, different classes of shapes have been considered [5, 6], shape models of human faces and human bodies are of special interest due to their immense applicability in human–machine interaction.”
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (2019, background): “[8] is an early work that learns a morphable model of dolphins from 2D keypoints and segmentation masks.”
Learning Category-Specific Deformable 3D Models for Object Reconstruction (2017, background): “Index Terms—Object reconstruction, 3D shape modeling, viewpoint estimation, scene understanding Ç”
Learning an efficient model of hand shape variation from depth images (2015, background): “Cashman and Fitzgibbon [9] demonstrate that morphable models using subdivision surfaces can be learned from extremely limited data (30 silhouette images).”
3D Menagerie: Modeling the 3D Shape and Pose of Animals (2016, methodology): “ras, 4 cows, 3 hippos. We estimated a scaling factor so animals from different manufacturers were comparable in size. Like previous 3D human datasets [26], and methods that create animals from images [9, 17], we collected a set of 36 hand-clicked 3 Figure 3: Location of 36 keypoints on the template ”