Deep Imbalanced Learning for Face Recognition and Attribute Prediction

By Chen Huang, Yining Li, Chen Change Loy, Xiaoou Tang. 2018

337 citations recorded by the source corpus.

Data for face analysis often exhibit highly-skewed class distribution, i.e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances. To mitigate this issue, contemporary deep learning methods typically follow classic strategies such as class re-sampling or cost-sensitive training. In this paper, we conduct extensive and systematic experiments to validate the effectiveness of these classic schemes for representation learning on class-imbalanced data. We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain inter-cluster margins both within and between classes. This tight constraint effectively reduces the class imbalance inherent in the local data neighborhood, thus carving much more balanced class boundaries locally. We show that it is easy to deploy angular margins between the cluster distributions on a hypersphere manifold. Such learned Cluster-based Large Margin Local Embedding (CLMLE), when combined with a simple k-nearest cluster algorithm, shows significant improvements in accuracy over existing methods on both face recognition and face attribute prediction tasks that exhibit imbalanced class distribution.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Long-Tailed Class Incremental Learning (2022, background): “However, data sampled from the real world often follows a long-tailed distribution [14,24,40,49] in which some classes have many more samples than others.”
Decoupling Representation and Classifier for Long-Tailed Recognition (2019, background): “The loss can vary at class-level for matching a given data distribution and improving the generalization of tail classes (Cui et al., 2019; Khan et al., 2017; Cao et al., 2019; Khan et al., 2019; Huang et al., 2019).”
Rethinking the Value of Labels for Improving Class-Imbalanced Learning (2020, background): “In addition, cost-sensitive re-weighting schemes [6, 22, 23, 27, 52] are proposed to (dynamically) adjust weights during training for different classes or even different samples.”
Multi-label Contrastive Predictive Coding (2020, background): “When the dataset has extremely imbalanced classes, higher weights are given to classes with less frequency [21, 22, 52] or classes with less effective samples [11].”
The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation (2020, background): “Two major approaches tackling long-tail problems are sampling [4],[5], [17],[37] and loss re-weighting [11],[19],[20].”
Parametric Contrastive Learning (2021, methodology): “However, with large-scale data, re-weighting makes CNNs difficult to optimize during training [23, 24].”
The Devil of Face Recognition is in the Noise (2018, background): “Beyond existing eﬀorts of developing sophisticated losses and CNN architectures, our study has investigated the problem of face recognition from the data perspective.”
Past, Present, and Future of Face Recognition: A Review (2020, background): “[146] 2019 ResNet-64 1 CLMLE Loss CASIA WebFace (494k, 10k) 99.”
ELF: An Early-Exiting Framework for Long-Tailed Classification (2020, background): “Reweighting based approaches assign a larger weight to rare class examples and a smaller weight to the majority ones [6, 18, 19, 20, 21, 22].”
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss (2019, background): “The vanilla scheme re-weights classes proportionally to the inverse of their frequency [21, 22, 55].”
Balanced Meta-Softmax for Long-Tailed Visual Recognition (2020, background): “, cost-sensitive learning [31, 11, 12, 24], assigns a cost weight to the loss of each class.”
DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data (2021, background): “Although the imbalanced data problem strongly affects both deep learning models [12] and their shallow counterparts, there”
Seesaw Loss for Long-Tailed Instance Segmentation (2020, methodology): “Loss re-weighting methods adopt different re-weighting strategies [2, 7, 20, 28, 44] to adjust the loss of different classes based on each class’s statistics [2, 7].”
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification (2020, background): “On the other hand, various cost-sensitive loss functions have been proposed in the literature [24, 47, 7, 27, 19, 2, 5].”
Unequal-Training for Deep Face Recognition With Long-Tailed Noisy Data (2019, neutral): “78 CLMLE [14] Large 76.”
Imbalance Problems in Object Detection: A Review (2019, background): “face recognition) are also affected by imbalance [159], [160], [161].”
Delving into Deep Imbalanced Regression (2021, positive): “This phenomenon poses great challenges for deep recognition models, and has motivated many prior techniques for addressing data imbalance (Cao et al., 2019; Cui et al., 2019; Huang et al., 2019; Liu et al., 2019; Tang et al., 2020).”
Overcoming Classifier Imbalance for Long-Tail Object Detection With Balanced Group Softmax (2020, methodology): “Also, loss re-weighting methods [5, 2, 18, 23, 33] can take effect in a similar way.”