IExpressNet: Facial Expression Recognition with Incremental Classes

By Junjie Zhu, Bingjun Luo, Sicheng Zhao, Shihui Ying, Xibin Zhao, Yue Gao. 2020

22 citations recorded by the source corpus.

Existing methods on facial expression recognition (FER) are mainly trained in the setting when all expression classes are fixed in advance. However, in real applications, expression classes are becoming increasingly fine-grained and incremental. To deal with sequential expression classes, we can fine-tune or re-train these models, but this often results in poor performance or large computing resources consumption. To address these problems, we develop an Incremental Facial Expression Recognition Network (IExpressNet), which can learn a competitive multi-class classifier at any time with a lower requirement of computing resources. Specifically, IExpressNet consists of two novel components. First, we construct an exemplar set by dynamically selecting representative samples from old expression classes. Then, the exemplar set and new expression classes samples constitute the training set. Second, we design a novel center-expression-distilled loss. As for facial expression in the wild, center-expression-distilled loss enhances the discriminative power of the deeply learned features and prevents catastrophic forgetting. Extensive experiments are conducted on two large-scale FER datasets in the wild, RAF-DB and AffectNet. The results demonstrate the superiority of the proposed method as compared to state-of-the-art incremental learning approaches.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Meta & Continual Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Liu Liu
5. Dacheng Tao
6. Zhen Wang
7. Yajing Kong
8. Jiaxian Guo

In Experimental and Cognitive Psychology

1. Dacheng Tao
2. De-chuan Zhan
3. Da-Wei Zhou
4. Liu Liu
5. Han-Jia Ye
6. Zhen Wang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Yajing Kong
5. Dacheng Tao
6. Yiqun Duan
7. Liu Liu
8. Zhen Wang

In Signal Processing

1. Dacheng Tao
2. De-chuan Zhan
3. Yiqun Duan
4. Da-Wei Zhou
5. Liu Liu
6. Han-Jia Ye
7. Zhen Wang

Citations in the corpus

• Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “IExpressNet [95] introduces representative expression memory while employing a novel Center-expression-distilled loss and having shown satisfactory performance for the task of facial expression recognition.”
• Continual Learning through Retrieval and Imagination (2022, background): “Unfortunately, most of the exiting methods fail to satisfy all these guidelines mentioned above (Chaudhry et al. 2021; Wang et al. 2020; Lopez-Paz et al. 2017; Wei et al. 2021; Riemer et al. 2019; Kong et al. 2021; Schwarz et al. 2018; Zhu et al. 2020).”
• Co-Transport for Class-Incremental Learning (2021, methodology): “Following [17, 51, 69], we select 6 basic expressions (without neutral) as the experimental data.”
• SIN: Semantic Inference Network for Few-Shot Streaming Label Learning (2022, background): “As new labels usually emerge from open and dynamic environments [1]–[3], [7], [21], streaming label learning is proposed to facilitate the learning system with the capability of modeling new labels effectively [8], [9].”