Probabilistic Face Embeddings

By Yichun Shi, Anil K. Jain, Nathan D. Kalka. IEEE International Conference on Computer Vision, 2019

371 citations recorded by the source corpus.

Embedding methods have achieved success in face recognition by comparing facial features in a latent semantic space. However, in a fully unconstrained face setting, the facial features learned by the embedding model could be ambiguous or may not even be present in the input face, leading to noisy representations. We propose Probabilistic Face Embeddings (PFEs), which represent each face image as a Gaussian distribution in the latent space. The mean of the distribution estimates the most likely feature values while the variance shows the uncertainty in the feature values. Probabilistic solutions can then be naturally derived for matching and fusing PFEs using the uncertainty information. Empirical evaluation on different baseline models, training datasets and benchmarks show that the proposed method can improve the face recognition performance of deterministic embeddings by converting them into PFEs. The uncertainties estimated by PFEs also serve as good indicators of the potential matching accuracy, which are important for a risk-controlled recognition system.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/53ea629b5933430eafa98692edf39ab2bd737d09

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

AdaFace: Quality Adaptive Margin for Face Recognition (2022, methodology): “To perform quality aware fusion, probabilistic approaches have been proposed to predict uncertainty in FR representation [2, 17, 26, 29].”
MagFace: A Universal Representation for Face Recognition and Quality Assessment (2021, methodology): “Recently, confidence-aware methods [29, 7] propose to represent each face image as a Gaussian distribution in the latent space, where the mean of the distribution estimates the most likely feature values while the variance shows the uncertainty in the feature values.”
Uncertainty-Aware Learning for Zero-Shot Semantic Segmentation (2020, methodology): “Besides serving as prediction conﬁdence as in the above methods, uncertainty is also modeled in distributional feature representation for applications like person ReID [58] and facial recognition [5, 49] to achieve robust feature learning.”
CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition (2020, methodology): “, Multicolumn [35], DCN [34], Adacos [38], P2SGrad [39], PFE [25] and MV-Arc-Softmax [31] on IJB-B and IJB-C 1:1 verification, our method again achieves the best performance.”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “However, we found that there are very few studies on probabilistic embedding strategies [501], [502] to quantify uncertainties.”
Data Uncertainty Learning in Face Recognition (2020, methodology): “Thereinto PFE [35], is the first work to consider data uncertainty in face recognition task.”
Probabilistic Embeddings for Cross-Modal Retrieval (2021, background): “[37] proposed the Hedged Instance Embedding (HIB) to handle the one-to-many correspondences for metric learning, while other works apply probabilistic embeddings to face understanding [43, 3], 2D-to-3D pose estimation [46], speaker diarization [44], and prototype embeddings [42].”
AdvHat: Real-World Adversarial Attack on ArcFace Face ID System (2019, methodology): “com/deepinsight/ insightface/wiki/Model-Zoo, accessed October 13, 2020) and from the repository of the author of Probabilistic Face Embeddings (PFE) [28] (https://github.”
WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition (2021, methodology): “Different model ensemble and post-processing [50] could be adopted for higher performance under these protocols.”
Prime-Aware Adaptive Distillation (2020, background): “, face recognition [30], semantic segmentation [15,16], object detection [7,17] and person re-identification(re-ID) [38].”
Discriminability Distillation in Group Representation Learning (2020, neutral): “PFE [45] 89.”