From Facial Expression Recognition to Interpersonal Relation Prediction

By Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang. International Journal of Computer Vision, 2016

310 citations recorded by the source corpus.

Interpersonal relation defines the association, e.g., warm, friendliness, and dominance, between two or more people. We investigate if such fine-grained and high-level relation traits can be characterized and quantified from face images in the wild. We address this challenging problem by first studying a deep network architecture for robust recognition of facial expressions. Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data. While conventional supervised training requires datasets with complete labels (e.g., all samples must be labeled with gender, age, and expression), we show that this requirement can be relaxed via a novel attribute propagation method. The approach further allows us to leverage the inherent correspondences between heterogeneous attribute sources despite the disparate distributions of different datasets. With the network we demonstrate state-of-the-art results on existing facial expression recognition benchmarks. To predict inter-personal relation, we use the expression recognition network as branches for a Siamese model. Extensive experiments show that our model is capable of mining mutual context of faces for accurate fine-grained interpersonal prediction.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/07d6122969754ce4f6e10b5a5f6a9dc77b60f75c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Shan Li
2. Weihong Deng
3. Naoyuki Kubota
4. Zhaojie Ju
5. Jing Li
6. Antonio Liotta
7. Kan Jin
8. Weifeng Ge

In Experimental and Cognitive Psychology

1. Weihong Deng
2. Shan Li

Citations in the corpus

• A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, background): “database for Expression in-the-Wild (ExpW) [85] contains 91,793 facial images, manually labelled with 7 facial expressions.”
• Attention mechanism-based CNN for facial expression recognition (2020, methodology): “The comparison results of our method and some representative methods [37][39][41][24][34] on this dataset are listed in Table 5, which indicate that our method performs better than most of the methods except Zhang et al. [41].”
• Deep Facial Expression Recognition: A Survey (2018, background): “ExpW [46] 91,793 images N/A Internet P & S seven basic expressions http://mmlab.”