Facial Expression Recognition by De-expression Residue Learning

By Huiyuan Yang, Umur Aybars Ciftci, Lijun Yin. 2018

384 citations recorded by the source corpus.

A facial expression is a combination of an expressive component and a neutral component of a person. In this paper, we propose to recognize facial expressions by extracting information of the expressive component through a de-expression learning procedure, called De-expression Residue Learning (DeRL). First, a generative model is trained by cGAN. This model generates the corresponding neutral face image for any input face image. We call this procedure de-expression because the expressive information is filtered out by the generative model; however, the expressive information is still recorded in the intermediate layers. Given the neutral face image, unlike previous works using pixel-level or feature-level difference for facial expression classification, our new method learns the deposition (or residue) that remains in the intermediate layers of the generative model. Such a residue is essential as it contains the expressive component deposited in the generative model from any input facial expression images. Seven public facial expression databases are employed in our experiments. With two databases (BU-4DFE and BP4D-spontaneous) for pre-training, the DeRL method has been evaluated on five databases, CK+, Oulu-CASIA, MMI, BU-3DFE, and BP4D+. The experimental results demonstrate the superior performance of the proposed method.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Shan Li
5. Weihong Deng
6. Yue Gao
7. Tengfei Song
8. Junjie Zhu

In Experimental and Cognitive Psychology

1. Weihong Deng
2. Shan Li
3. Tengfei Song
4. De-chuan Zhan
5. Da-Wei Zhou
6. Han-Jia Ye
7. Yuru Wang
8. Qiang Ji

Citations in the corpus

• Co-Transport for Class-Incremental Learning (2021, methodology): “Following [17, 51, 69], we select 6 basic expressions (without neutral) as the experimental data.”
• IExpressNet: Facial Expression Recognition with Incremental Classes (2020, methodology): “Following the previous works [27, 71, 75] for FER, we select 6 basic expressions as the experimental data of this paper.”
• Attention mechanism-based CNN for facial expression recognition (2020, methodology): “The comparison results of our method and some representative methods [37][39][41][24][34] on this dataset are listed in Table 5, which indicate that our method performs better than most of the methods except Zhang et al.”
• Deep Facial Expression Recognition: A Survey (2018, background): “tage of GANs is that identity variations can be explicitly disentangled by generating the corresponding neutral face image [105] or synthesizing different expressions while pre-”
• Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition (2020, background): “Pre-trained models can be constructed from non-facial data([4]) and other facial datasets[50].”
• Generative Adversarial Networks for Face Generation: A Survey (2022, background): “Yang et al. [144] proposed a DeRL (de-expression residue learning procedure) to recognize facial expressions by extracting information of the expressive component.”