FaceNet: A unified embedding for face recognition and clustering
By Florian Schroff, Dmitry Kalenichenko, James Philbin. 2015
15,122 citations recorded by the source corpus.
Despite significant recent advances in the field of face recognition [10, 14, 15, 17], implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings asfeature vectors. Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches. To train, we use triplets of roughly aligned matching / non-matching face patches generated using a novel online triplet mining method. The benefit of our approach is much greater representational efficiency: we achieve state-of-the-artface recognition performance using only 128-bytes perface. On the widely used Labeled Faces in the Wild (LFW) dataset, our system achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves 95.12%. Our system cuts the error rate in comparison to the best published result [15] by 30% on both datasets.
Subfields: Computer Vision and Pattern Recognition, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Yi Yang
- Andrew Zisserman
- Jitendra Malik
In Signal Processing
- Andrew Zisserman
- Yoshua Bengio
- James R. Glass
- Weihong Deng
- Kristen Grauman
- Dong Yu
- Wei-Ning Hsu
- Jinyu Li
Citations in the corpus
- Rethinking the Inception Architecture for Computer Vision (2015, methodology): “This has made it feasible to utilize Inception networks in big-data scenarios[17], [13], where huge amount of data needed to be processed at reasonable cost or scenarios where memory or computational capacity is inherently limited, for example in mobile vision settings.”
- A Simple Framework for Contrastive Learning of Visual Representations (2020, neutral): “Normalized cross entropy loss with adjustable temperature works better than alternatives We compare the NT-Xent loss against other commonly used contrastive loss functions, such as logistic loss (Mikolov et al., 2013), and margin loss (Schroff et al., 2015).”
- Self-Supervised Learning of Video-Induced Visual Invariances (2019, methodology): “Following [12, 33], to scale this approach to hundreds of millions of images (frames), we employ a triplet loss [52] encouraging augmentations of the same image to be close and augmentations of different images to be far apart.”
- FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “To date, numerous large scale face image datasets [22, 31, 13, 69, 37, 24, 41, 68, 15, 27, 46, 8, 39] have been proposed and fostered research and development for automated face detection [35, 21], alignment [66, 44], recognition [57, 49], generation [67, 5, 26, 58], modification [3, 32, 19], and at”
- Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “Recently, focusing on hard-negatives has been shown to be effective in learning joint embeddings [8, 44, 56, 60].”
- Representation Learning with Contrastive Predictive Coding (2018, methodology): “For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
- Revisiting IM2GPS in the Deep Learning Era (2017, background): “Image retrieval using learned, deep representations is useful to a wide range of tasks such as product ranking [4, 12], sketch based image retrieval [25], face recognition [31, 26], cross-view localization [18, 37, 34] and scene retrieval [33, 2, 23, 8].”
- Spatial Transformer Networks (2015, background): “Though not a recent invention, we now see a cornucopia of CNN-based models achieving state-of-the-art results in classification [19, 27, 34], localisation [30, 36], semantic segmentation [24], and action recognition [12, 31] tasks, amongst others.”
- Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “An important technical point on metric learning for face recognition is normalization [35, 22, 43], which we also utilize in this work.”
- Conditional Image Generation with PixelCNN Decoders (2016, methodology): “The network was trained with a triplet loss function [23] that ensured that the embedding h produced for an image x of a specific person was closer to the embeddings for all other images of the same person than it was to any embedding of another person.”
- VAIN: Attentional Multi-agent Predictive Modeling (2017, neutral): “Deep Neural Networks (DNNs) have had much success in machine perception e.g. Computer Vision [1, 2, 3], Natural Language Processing [4] and Speech Recognition [5, 6].”
- On Mutual Information Maximization for Representation Learning (2019, background): “Secondly, it elucidates the importance of appropriately choosing the negative samples, which is indeed a critical component in deep metric learning based on triplet losses (Norouzi et al., 2012; Schroff et al., 2015).”
- Unsupervised Embedding Learning via Invariant and Spreading Instance Feature (2019, background): “In particular, several sampling strategies are widely investigated to improve the performance, such as hard mining [16], semihard mining [35], smart mining [13] and so on.”
- Revisiting Self-Supervised Visual Representation Learning (2019, methodology): “[8] proposes to use the triplet loss [40, 18] in order to scale this pretext task to a large number of images (hence, classes) present in the ImageNet dataset.”
- Unsupervised Pretraining Transfers Well Across Languages (2020, background): “for speech and we focus on those based on contrastive learning [7, 14, 15].”
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, methodology): “The FaceNet model is a state of the art face recognition model [25].”
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019, methodology): “In order to fine-tune BERT / RoBERTa, we create siamese and triplet networks (Schroff et al., 2015) to update the weights such that the produced sentence embeddings are semantically meaningful and can be compared with cosine-similarity.”
- Learning Transferable Architectures for Scalable Image Recognition (2017, background): “The ImageNet re-sults are particularly important because many state-of-the-art computer vision problems (e.g., object detection [18], face detection [29], image localization [39]) derive image features or architectures from ImageNet classification models.”