Learning Social Relation Traits from Face Images

By Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang. 2015

177 citations recorded by the source corpus.

Social relation defines the association, e.g., warm, friendliness, and dominance, between two or more people. Motivated by psychological studies, we investigate if such fine grained and high-level relation traits can be characterised and quantified from face images in the wild. To address this challenging problem we propose a deep model that learns a rich face representation to capture gender, expression, head pose, and age-related attributes, and then performs pairwise-face reasoning for relation prediction. To learn from heterogeneous attribute sources, we formulate a new network architecture with a bridging layer to leverage the inherent correspondences among these datasets. It can also cope with missing target attribute labels. Extensive experiments show that our approach is effective for fine-grained social relation learning in images and videos.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “Face attribute recognition is a task to classify various human attributes such as gender, race, age, emotions, expressions or other facial traits from facial appearance [31, 25, 74, 37].”
A Domain Based Approach to Social Relation Recognition (2017, methodology): “Other similar models can also be considered, such as Siamese-like architectures used for face modeling [41, 38, 39], and multichannel CNNs used for person identification [40].”
FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply (2020, unknown): “For EXPW, we use the true bounding box associated with the dataset to create aligned faces ﬁrst, and only pick the images that are faces with conﬁdence larger than 0.6.”
Deep Facial Expression Recognition: A Survey (2018, unknown): “88.9 (87.7) FER2013 Tang 13 [132] L2-SVM loss Training Set: 28,709 Validation Set: 3,589 Test Set: 3,589 7 Test: 71.2 Devries et al. 14 [34] multitask network 7 Validation+Test: 67.21 Zhang et al. 15 [133] bridging layer 7 Test: 75.10 Guo et al. 16 [134] triplet-based loss 7 Test: 71.33 Kim et al. 1”
From Facial Expression Recognition to Interpersonal Relation Prediction (2016, methodology): “In comparison to our earlier version of this work [93], we present a more principle and unified way of addressing the heterogeneous data problem using the MRF-based”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation (2021, background): “The goal of face attribute recognition is to classify various human attributes such as gender, race, age, emotions, expressions or other facial traits from facial appearance [31, 24, 75, 37].”