Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

By Yingruo Fan, Jacqueline C. K. Lam, Victor O. K. Li. 2020

60 citations recorded by the source corpus.

The intensity estimation of facial action units (AUs) is challenging due to subtle changes in the person's facial appearance. Previous approaches mainly rely on probabilistic models or predefined rules for modeling co-occurrence relationships among AUs, leading to limited generalization. In contrast, we present a new learning framework that automatically learns the latent relationships of AUs via establishing semantic correspondences between feature maps. In the heatmap regression-based network, feature maps preserve rich semantic information associated with AU intensities and locations. Moreover, the AU co-occurring pattern can be reflected by activating a set of feature channels, where each channel encodes a specific visual pattern of AU. This motivates us to model the correlation among feature channels, which implicitly represents the co-occurrence relationship of AU intensity levels. Specifically, we introduce a semantic correspondence convolution (SCC) module to dynamically compute the correspondences from deep and low resolution feature maps, and thus enhancing the discriminability of features. The experimental results demonstrate the effectiveness and the superior performance of our method on two benchmark datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Pascale Fung
2. Zhaojiang Lin
3. Andrea Madotto
4. Sumanth Dathathri
5. Etsuko Ishii

Citations in the corpus

• Plug-and-Play Conversational Models (2020, background): “, 2020), emotions (Li et al., 2017; Rashkin et al., 2019; Zhou et al., 2018; Fan et al., 2020; Li et al., 2020), diversity (Li et al.”