Fine-Grained Head Pose Estimation Without Keypoints

By Nataniel Ruiz, Eunji Chong, James M. Rehg. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017

622 citations recorded by the source corpus.

Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment. Traditionally head pose is computed by estimating some keypoints from the target face and solving the 2D to 3D correspondence problem with a mean human head model. We argue that this is a fragile method because it relies entirely on landmark detection performance, the extraneous head model and an ad-hoc fitting step. We present an elegant and robust way to determine pose by training a multi-loss convolutional neural network on 300W-LP, a large synthetically expanded dataset, to predict intrinsic Euler angles (yaw, pitch and roll) directly from image intensities through joint binned pose classification and regression. We present empirical tests on common in-the-wild pose benchmark datasets which show state-of-the-art results. Additionally we test our method on a dataset usually used for pose estimation using depth and start to close the gap with state-of-the-art depth pose methods. We open-source our training and testing code as well as release our pre-trained models.

Subfields: Computer Vision and Pattern Recognition, Dermatology

View Paper: https://www.semanticscholar.org/paper/23dcfda130aada27c158c0b5f394cac489c9c795

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Dermatology

Citations in the corpus

LAEO-Net: Revisiting People Looking at Each Other in Videos (2019, background): “There are several works that model the head orientation [27] or the eye gaze [23].”
Dual Attention Guided Gaze Target Detection in the Wild (2021, methodology): “We compare the proposed 3D gaze estimation module against other state-of-the-art methods [39, 22] on the Gaze360 dataset [22] to evaluate the performance of gaze estimation in the wild.”
Blind Face Restoration via Deep Multi-scale Component Dictionaries (2020, methodology): “We utilize DeepPose [32] and Face to recognize their poses and expressions (i.”
FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping (2019, methodology): “2, 5, 6 [36] Nataniel Ruiz, Eunji Chong, and James M Rehg.”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, methodology): “We measure the editing magnitude using trait-specific classifiers (HopeNet [55] for pose and the classifier from Lin et al.”
SimSwap: An Efficient Framework For High Fidelity Face Swapping (2020, methodology): “We also use the pose estimator [32] to estimate the postures in the generated frames and the original frames, and we calculate their averaged L2 distance.”
FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image (2019, methodology): “Several approaches only utilize a single RGB image for pose estimation [6, 35, 32, 22, 31].”
Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs (2020, methodology): “To further study the possibility of unintended usage of pretrained GANs, we employ our framework to achieve head pose manipulation by leveraging pre-trained HopeNet [24] as the task model.”
Gaze360: Physically Unconstrained Gaze Estimation in the Wild (2019, background): “[19]; Static - the backbone model, ResNet-18, and two final layers to compute the prediction; TRN - a version of Temporal Relation Network [33] where the features of frames at fixed windows around time t are concatenated before averaging the predictions of the temporal windows; LSTM - refers to the ”
Detecting Attended Visual Targets in Video (2020, background): “Prior work in this area has leveraged the ability to analyze head orientation [45, 16, 11] to infer children’s attention and have developed solutions for specific settings [41, 20, 5].”
Energy-Based Models for Deep Probabilistic Regression (2020, methodology): “This approach has previously been applied to both age estimation [48, 42, 60] and head-pose estimation [49, 59].”
Third Time's the Charm? Image and Video Editing with StyleGAN3 (2022, methodology): “We apply HopeNet [59] for pose, Rothe et al .”
One Shot Face Swapping on Megapixels (2021, methodology): “As for pose and expression errors, an open-sourced pose estimator [47] and a 3D facial model [13] are used to extract pose and expression vectors.”