DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

By Riza Alp Güler, George Trigeorgis, Epameinondas Antonakos, Patrick Snape, Stefanos P. Zafeiriou, Iasonas Kokkinos. 2016

206 citations recorded by the source corpus.

In this paper we propose to learn a mapping from image pixels into a dense template grid through a fully convolutional network. We formulate this task as a regression problem and train our network by leveraging upon manually annotated facial landmarks in-the-wild. We use such landmarks to establish a dense correspondence field between a three-dimensional object template and the input image, which then serves as the ground-truth for training our regression system. We show that we can combine ideas from semantic segmentation with regression networks, yielding a highly-accurate quantized regression architecture. Our system, called DenseReg, allows us to estimate dense image-to-template correspondences in a fully convolutional manner. As such our network can provide useful correspondence information as a stand-alone system, while when used as an initialization for Statistical Deformable Models we obtain landmark localization results that largely outperform the current state-of-the-art on the challenging 300W benchmark. We thoroughly evaluate our method on a host of facial analysis tasks, and demonstrate its use for other correspondence estimation tasks, such as the human body and the human ear. DenseReg code is made available at http://alpguler.com/DenseReg.html along with supplementary materials.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

DensePose: Dense Human Pose Estimation in the Wild (2018, methodology): “urn to the task of training a deep network that predicts dense correspondences between image pixels and surface points. Such a task was recently addressed in the Dense Regression (DenseReg) system of [13] through a fully-convolutional network architecture [4]. In this work, we introduce improved arc”
End-to-End Recovery of Human Shape and Pose (2017, background): “DenseReg [13] similarly outputs a dense correspondence map for human bodies.”
Multilinear Autoencoder for 3D Face Model Learning (2018, methodology): “CNNs have also been used for shape regression of 3D faces in-the-wild [13].”
Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation (2017, methodology): “Alternatively, other methods [16, 43] proposed to generate training data by employing existing reconstruction algorithms and regarding their results as ground-truth labels.”
Single Image 3D Hand Reconstruction with Mesh Convolutions (2019, methodology): “In terms of applications beyond computer graphics, our system can be modified to solve the problems of hand pose estimation [29, 30, 39], fingertip detection [35], or dense correspondence computation [13, 14].”
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction (2017, background): “The generation of faces from attributes [30] and dense shape regression [16] have also been studied.”
DeepHuman: 3D Human Reconstruction From a Single Image (2019, background): “The research can be categorized into sparse 2D parsing (2D skeleton estimation) [70, 50, 5, 47], sparse 3D parsing (3D skeleton estimation) [41, 49, 53, 78, 59, 62, 44, 74], dense 2D parsing [18, 17] and dense 3D parsing (shape and pose estimation).”
3FabRec: Fast Few-Shot Face Alignment by Reconstruction (2019, unknown): “05 DenseReg+MDM [1] 52.”
RetinaFace: Single-stage Dense Face Localisation in the Wild (2019, methodology): “Nevertheless, in this paper we refer to a broader deﬁnition of face localisation which includes face detection [39], face alignment [13], pixel-wise face parsing [48] and 3D dense correspondence regression [2, 12].”
BodyNet: Volumetric Inference of 3D Human Body Shapes (2018, background): “DenseReg [47] and DensePose [48] estimate image-to-surface correspondences, while [33] outputs quantized depth maps for SMPL bodies.”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “Learning-based Approaches: In addition to optimization-based reconstruction approaches, there are many learning-based methods [72, 41, 27, 20, 56].”
High-Resolution Representations for Labeling Pixels and Regions (2019, unknown): “DReg + MDM [35] ResNet101 - - 52.”
Look at Boundary: A Boundary-Aware Face Alignment Algorithm (2018, background): “83 DenseReg + MDM [1] 0.”
Learning an animatable detailed 3D face model from in-the-wild images (2020, methodology): “However, all these methods require explicit 3D supervision, which is provided either by an optimization-based model fitting [Feng et al. 2018b; Güler et al. 2017; Jackson et al. 2017; Wei et al. 2019] or by synthetic data generated by sampling a statistical face model [Dou et al.”
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network (2018, methodology): “ent from other 2D Joint 3D Face Reconstruction and Dense Alignment 7 representations such as Projected Normalized Coordinate Code (PNCC)[11,14], an ordinary depth image[17] or quantized UV coordinates[28], which only reserve the information of visible face region in the input image. Our proposed pos”
EMOCA: Emotion Driven Monocular Face Capture and Animation (2022, methodology): “Model-free approaches directly regress 3D meshes [18,20,28,33,40,70,74,79,93,95,97] or voxels [38] from an image, or optimize a Signed Distance Function (SDF) [61] to fit a face image.”