AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations

By Xiao Zhang, Rui Zhao, Yu Qiao, Xiaogang Wang, Hongsheng Li. 2019

252 citations recorded by the source corpus.

The cosine-based softmax losses and their variants achieve great success in deep learning based face recognition. However, hyperparameter settings in these losses have significant influences on the optimization path as well as the final recognition performance. Manually tuning those hyperparameters heavily relies on user experience and requires many training tricks. In this paper, we investigate in depth the effects of two important hyperparameters of cosine-based softmax losses, the scale parameter and angular margin parameter, by analyzing how they modulate the predicted classification probability. Based on these analysis, we propose a novel cosine-based softmax loss, AdaCos, which is hyperparameter-free and leverages an adaptive scale parameter to automatically strengthen the training supervisions during the training process. We apply the proposed AdaCos loss to large-scale face verification and identification datasets, including LFW, MegaFace, and IJB-C 1:1 Verification. Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy. Our method outperforms state-of-the-art softmax losses on all the three datasets.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation (2022, methodology): “We implemented some methods in the face recognition community that might be relevant to our work, including (1) Adaptive-margin-based methods (CurricularFace [17], AdaCos [41], ElasticFace [3], AMRLoss [42]), such as adjusting the margin value according to the intra/inter-class angles.”
MagFace: A Universal Representation for Face Recognition and Quality Assessment (2021, background): “More recently, AdaptiveFace [19], AdaCos [49] and FairLoss [18] introduce adaptive margin strategy to automatically tune hyperparameters and generate more effective supervisions during training.”
Circle Loss: A Unified Perspective of Pair Similarity Optimization (2020, background): “It is because in the high dimensional feature space, randomized features are prone to be far away from each other [40, 7].”
Mitigating Neural Network Overconfidence with Logit Normalization (2022, methodology): “…the literature, normalization has been widely used in metric learning (Sohn, 2016; Wu et al., 2018; van den Oord et al., 2018), face recognition (Ranjan et al., 2017; Liu et al., 2017; Wang et al., 2017; 2018; Deng et al., 2019; Zhang et al., 2019), and self-supervised learning (Chen et al., 2020).”
SphereFace Revived: Unifying Hyperspherical Face Recognition (2021, background): “[36] and [37] consider adaptive schemes to set the radius of the hypersphere and the margin parameter, respectively.”
A Survey of Deep Learning-Based Object Detection (2019, background): “The cosine-based softmax losses [129]–[132] achieve great success in deep learning based face recognition.”
Regularizing Class-Wise Predictions via Self-Knowledge Distillation (2020, methodology): “We compare our method with prior regularization methods such as the state-of-the-art angular-margin based methods [5, 58], entropy regularization [13, 32, 36, 43] and self-distillation methods [53, 57].”
AdaFace: Quality Adaptive Margin for Face Recognition (2022, background): “Many studies have introduced an element of adaptiveness in the training objective for either hard sample mining [18, 36], scheduling difficulty during training [13, 31], or finding optimal hyperparameters [41].”
Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data (2020, methodology): “More importantly, the temperature scaling is generally used as a numerical trick for learning a better embedding [40], softening the prediction [15], or calibrating the confidence [9].”
CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition (2020, methodology): “For example, explicit margin, i.e. , Cos-Face [30], Sphereface [14], ArcFace [8], and implicit margin, i.e. , Adacos [38], supplement the original softmax function to enforce greater intra-class compactness and inter-class discrepancy, which result in more discriminate features.”
ElasticFace: Elastic Margin Loss for Deep Face Recognition (2021, unknown): “73 Dynamic-AdaCos[31] (CVPR2019) clean MS1M [7], [31] + CASIA [29] 99.”