Deeply learned face representations are sparse, selective, and robust

By Yi Sun, Xiaogang Wang, Xiaoou Tang. 2014

955 citations recorded by the source corpus.

This paper designs a high-performance deep convolutional network (DeepID2+) for face recognition. It is learned with the identification-verification supervisory signal. By increasing the dimension of hidden representations and adding supervision to early convolutional layers, DeepID2+ achieves new state-of-the-art on LFW and YouTube Faces benchmarks. Through empirical studies, we have discovered three properties of its deep neural activations critical for the high performance: sparsity, selectiveness and robustness. (1) It is observed that neural activations are moderately sparse. Moderate sparsity maximizes the discriminative power of the deep net as well as the distance between images. It is surprising that DeepID2+ still can achieve high recognition accuracy even after the neural responses are binarized. (2) Its neurons in higher layers are highly selective to identities and identity-related attributes. We can identify different subsets of neurons which are either constantly excited or inhibited when different identities or attributes are present. Although DeepID2+ is not taught to distinguish attributes during training, it has implicitly learned such high-level concepts. (3) It is much more robust to occlusions, although occlusion patterns are not included in the training set.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

FaceNet: A unified embedding for face recognition and clustering (2015, methodology): “Previous face recognition approaches based on deep networks use a classification layer [15, 17] trained over a set of known face identities and then take an intermediate bottleneck layer as a representation used to generalize recognition beyond the set of identities used in training.”
Why Should We Add Early Exits to Neural Networks? (2020, methodology): “Apart from Inception, similar approaches have been successfully applied to, among others, face recognition [61], object detection [14], image segmentation [15], pose estimation [25], saliency detection [41], metric learning [71], visual attention prediction [69], and super resolution [34, 65].”
Faceless Person Recognition: Privacy Implications in Social Media (2016, background): “Person recognition systems like [20, 27, 26] all rely on undisclosed training data in the order of millions of training samples.”
Empirical Evaluation of Rectified Activations in Convolutional Network (2015, background): “In all of these non-saturated activation functions, the most notable one is rectiﬁed linear unit (ReLU) (Nair & Hinton, 2010; Sun et al., 2014).”
Large-Margin Softmax Loss for Convolutional Neural Networks (2016, background): “…boosted the state-of-the-art performance in many visual classiﬁcation tasks such as object recognition, (Krizhevsky et al., 2012; Sermanet et al., 2014; He et al., 2015b;a), face veriﬁcation (Taigman et al., 2014; Sun et al., 2014; 2015) and hand-written digit recognition (Wan et al., 2013).”
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale (2015, methodology): “Other recent studies have considered both identification as well as verification results on LFW [2, 30, 28, 27].”
CosFace: Large Margin Cosine Loss for Deep Face Recognition (2018, background): “Recent works DeepID2+ [21] and DeepID3 [22] further explore the advanced network structures to boost performance.”
Naive-Deep Face Recognition: Touching the Limit of LFW Benchmark or Not? (2015, methodology): “Meanwhile, many sophisticated methods are developed and applied to recognition systems (e.g., joint Bayesian in [4, 2, 10, 12, 13], model ensemble in [10, 14], multi-stage feature in [10, 12], and joint identiﬁcation and veriﬁcation learning in [10, 13]).”
Person Recognition in Personal Photo Collections (2015, background): “Both settings have seen a recent shift from sophisticated classifiers based on hand-crafted features and metric learning approaches [20, 7, 5, 29, 26, 41, 1], towards methods based on deep learning [37, 36, 43, 33, 27, 38, 21].”
Robust Face Recognition via Multimodal Deep Face Representation (2015, result): “This is different from previous works that favor sparse features produced by ReLU [8], [9], [31].”
Discriminability Distillation in Group Representation Learning (2020, methodology): “ouTube Face dataset, compared with state-of-the-art methods and baseline methods Method Accuracy(%) Method Accuracy(%) Li et al. [32] 84.8 DeepFace [49] 91.4 FaceNet [44] 95.52 NAN [62] 95.72 DeepID2 [48] 93.20 QAN [37] 96.17 C-FAN [20] 96.50 Rao et al. [42] 96.52 Liu et al. [38] 96.21 Rao et al. [4”
Attention-Aware Deep Reinforcement Learning for Video Face Recognition (2017, neutral): “1 2015 DeepID2+ [34] 93.”
MegaFace: A Million Faces for Recognition at Scale (2015, methodology): “Recently, several papers reported identification results on the LFW dataset [3, 28, 26, 25].”
A Light CNN for Deep Face Representation With Noisy Labels (2015, methodology): “The results of our model on the LFW verification protocol are better than those of DeepFace[37], DeepID2+ [35], WebFace [43], VGG [27], CenterLoss [39] and SeetaFace [20] for a single net.”
When Face Recognition Meets with Deep Learning: An Evaluation of Convolutional Neural Networks for Face Recognition (2015, methodology): “Motivated by the very deep architectures of [20, 26], WebFace trains a much deeper CNN than those [24, 21, 25, 28] used for face recognition as shown in Table 1.”
A Comprehensive Study on Center Loss for Deep Face Recognition (2019, background): “Sun et al. (2015) enhanced the supervision by adding a fully-connected layer and loss functions to each convolutional layer.”