Neural Aggregation Network for Video Face Recognition

By Jiaolong Yang, Peiran Ren, Dongqing Zhang, Dong Chen, Fang Wen, Hongdong Li, Gang Hua. 2016

397 citations recorded by the source corpus.

This paper presents a Neural Aggregation Network (NAN) for video face recognition. The network takes a face video or face image set of a person with a variable number of face images as its input, and produces a compact, fixed-dimension feature representation for recognition. The whole network is composed of two modules. The feature embedding module is a deep Convolutional Neural Network (CNN) which maps each face image to a feature vector. The aggregation module consists of two attention blocks which adaptively aggregate the feature vectors to form a single feature inside the convex hull spanned by them. Due to the attention mechanism, the aggregation is invariant to the image order. Our NAN is trained with a standard classification or verification loss without any extra supervision signal, and we found that it automatically learns to advocate high-quality face images while repelling low-quality ones such as blurred, occluded and improperly exposed faces. The experiments on IJB-A, YouTube Face, Celebrity-1000 video face recognition benchmarks show that it consistently outperforms naive aggregation methods and achieves the state-of-the-art accuracy.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, methodology): “For face recognition, neural aggregation network [22] uses soft attention to adaptively aggregate frame features.”
VGGFace2: A Dataset for Recognising Faces across Pose and Age (2017, neutral): “[24] - 0.”
Deep Face Recognition: A Survey (2018, methodology): “The architecture of Alexnet, VGGNet, GoogleNet, ResNet, SENet.”
Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks (2017, background): “nose tip or eye centre, which provides rich geometric information for other face analysis tasks such as face recognition [55, 40, 37, 67], emotion estimation [69, 16, 57, 35] and 3D face reconstruction [15, 31, 26, 25, 48, 33, 19].”
Dual Attention Matching Network for Context-Aware Feature Sequence Based Person Re-identification (2018, background): “In [50, 69], fixed-dimension feature vectors are learned from variable length videos for face recognition and person ReID by attentive aggregation, respectively.”
Quality Aware Network for Set to Set Recognition (2017, background): “This mechanism helps feature generation part to focus on good samples and neglect ones, which benefits set-to-set recognition.”
L2-constrained Softmax Loss for Discriminative Face Verification (2017, methodology): “Most of the recent methods [29, 30, 24, 37] train a DCNN with softmax loss to learn these features which are later used either to directly compute the similarity score for a pair of faces or to train a discriminative metric embedding [27, 3].”
Feature Transfer Learning for Deep Face Recognition with Under-Represented Data (2018, methodology): “tion Method # 0:01 0:001 1 5 10 PAMs [26] 82 :6 65 2 84 0 92 5 94 6 DR-GAN [37] 83:1 69:9 90:1 95:3 { FF-GAN [48] 85:2 66:3 90:2 95:4 { TA [7] 93:9 { 92:8 { 98:6 TPE [31] 90:0 81:3 86:3 93:2 97:7 NAN [46] 94 :1 88 1 95 8 98 0 98 6 sfmx 91:5 77:4 92:4 96:4 97:3 sfmx + m-L 2 (Ours) 92:5 80:2 93:9 97:2”
Discriminability Distillation in Group Representation Learning (2020, methodology): “p-based recognition is how to dene an ecient indicator to select representatives from sets. To tackle such cases, previous methods aim at dening the quality or saliency for each element in a group [37,62,42,40]. The weight for each element can be automatically learned by self-attention. For example,”
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression (2019, background): “Face alignment plays an essential role in many face related applications such as face recognition [55, 43, 39, 70, 11], face frontalization [24, 60, 30] and 3D face reconstruction [16, 52, 37, 21].”
Towards Backward-Compatible Representation Learning (2020, background): “Common choices of the set function S include averaging and attention models [42].”
Feature Transfer Learning for Face Recognition With Under-Represented Data (2019, result): “We further combine media pooling (MP) and template adaptation (TA) [7] metric learning with our proposed method (FTL + MP + TA), and achieve consistently better results than state-of-the-art methods [46].”