HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition

By Rajeev Ranjan, Vishal M. Patel, Ramalingam Chellappa. 2019

1,297 citations recorded by the source corpus.

We present an algorithm for simultaneous face detection, landmarks localization, pose estimation and gender recognition using deep convolutional neural networks (CNN). The proposed method called, HyperFace, fuses the intermediate layers of a deep CNN using a separate CNN followed by a multi-task learning algorithm that operates on the fused features. It exploits the synergy among the tasks which boosts up their individual performances. Additionally, we propose two variants of HyperFace: (1) HyperFace-ResNet that builds on the ResNet-101 model and achieves significant improvement in performance, and (2) Fast-HyperFace that uses a high recall fast face detector for generating region proposals to improve the speed of the algorithm. Extensive experiments show that the proposed models are able to capture both global and local information in faces and performs significantly better than many competitive algorithms for each of these four tasks.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

In Signal Processing

Citations in the corpus

Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering (2017, methodology): “ayer, followed by task-speciﬁc (i.e., unshared) decoders that extract labels for each task (Devin et al., 2016; Dong et al., 2015; Huang et al., 2013; 2015; Jaderberg et al., 2017; Liu et al., 2015a; Ranjan et al., 2016; Wu et al., 2015; Zhang et al., 2014) (Figure 1a). This approach can be extended”
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification (2016, background): “HyperFace [28] simultaneously learns to perform face detection, landmarks localization, pose estimation and gender recognition.”
Generate to Adapt: Aligning Domains Using Generative Adversarial Networks (2017, methodology): “The landmarks used for alignment were obtained using the HyperFace approach [24].”
Edge Assisted Real-time Object Detection for Mobile Augmented Reality (2019, background): “Thanks to the idea of multitask learning, current CNN can further reuse the deep features inside the object bounding box for more fine-grained detection, such as instance segmentation [33], human key points detection [33], facial landmark detection [34], etc.”
S^3FD: Single Shot Scale-Invariant Face Detector (2017, methodology): “8 unlabelled faces (annotations will be released later and some examples are shown in the supplementary materials). Finally, we evaluate our face detector on FDDB against the state-of-the-art methods [1,6,7, 15,18,19,20,22,24,25,31,35,36,43,47,49,55,57,58]. The results are shown in Fig.7(a)and Fig.7”
PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing (2018, background): “The task can be typically treated as a deep multi-task learning problem [42].”
UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation (2019, background): “allyshared architecture setting [29]. Multi-task learning. Multi-task learning [3] has been applied in computer vision literature [38] for quite a long time for a varied set of tasks in consideration [27,8,9,16,43]. To realize this, a trivial direction is to formulate a multi-task loss function, whi”
Object Detection With Deep Learning: A Review (2018, background): “The outstanding performance of MTCNN, Conv3D and Hyperface proves the effectiveness of multi-task learning.”
Synthetic Data for Deep Learning (2019, background): “They report significantly improved results for face recognition and facial landmark detection with the OpenFace framework [16, 520] and state of the art models for face detection and alignment [692] and landmark detection [478].”
Convolutional neural network: a review of models, methodologies and applications to object detection (2019, background): “Ranjan [22] propounded a deep learning technique for understanding the faces.”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, background): “Recently, jointly modeling face detection with other auxiliary tasks including face alignment, pose estimation, and gender classiﬁcation, can achieve improved performance, owing to the extra abundant supervisory signals for learning a shared discriminative feature representation [96], [97].”
Truncated Back-propagation for Bilevel Optimization (2018, background): “This general setup subsumes multiple problems commonly encountered in the machine learning literature, such as multi-task learning [15, 16] and few-shot learning [17, 18, 19].”
JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method (2020, background): “On the other hand, the shallower layers have relatively more detailed local information as compared to their deeper counterparts [73].”
An All-In-One Convolutional Neural Network for Face Analysis (2016, methodology): “We select HyperFace [36] as a comparison baseline for the tasks of face detection, pose estimation, landmarks localization and gender recognition.”
Generating High-Quality Crowd Density Maps Using Contextual Pyramid CNNs (2017, methodology): “Inspired by cascaded multi-task learning [25, 4], Sindagi et al.”