Selective Transfer Machine for Personalized Facial Action Unit Detection

By Wen-Sheng Chu, Fernando De la Torre, Jeffrey F. Cohn. 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013

310 citations recorded by the source corpus.

Automatic facial action unit (AFA) detection from video is a long-standing problem in facial expression analysis. Most approaches emphasize choices of features and classifiers. They neglect individual differences in target persons. People vary markedly in facial morphology (e.g., heavy versus delicate brows, smooth versus deeply etched wrinkles) and behavior. Individual differences can dramatically influence how well generic classifiers generalize to previously unseen persons. While a possible solution would be to train person-specific classifiers, that often is neither feasible nor theoretically compelling. The alternative that we propose is to personalize a generic classifier in an unsupervised manner (no additional labels for the test subjects are required). We introduce a transductive learning method, which we refer to Selective Transfer Machine (STM), to personalize a generic classifier by attenuating person-specific biases. STM achieves this effect by simultaneously learning a classifier and re-weighting the training samples that are most relevant to the test subject. To evaluate the effectiveness of STM, we compared STM to generic classifiers and to cross-domain learning methods in three major databases: CK+, GEMEP-FERA and RU-FACS. STM outperformed generic classifiers in all.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/064cd41d323441209ce1484a9bba02a22b625088

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

Domain Generalization and Adaptation Using Low Rank Exemplar SVMs (2018, methodology): “We first investigate the state-of-the-art unsupervised domain adaptation methods, including Kernel Mean Matching (KMM) [21], Sampling Geodesic Flow (SGF) [2], Geodesic Flow Kernel (GFK) [3], Selective Transfer Machine (STM) [45], Domain Invariant Projection (DIP) [13], and”
Deep Model Based Domain Adaptation for Fault Diagnosis (2017, background): “The learning strategies of most published domain adaptation works can be roughly classified into two categories: instance transfer and subspace learning—the former reweights the source data according to the shared information contained in the target data, and then make a further analysis on the rewe”
Deep Hashing Network for Unsupervised Domain Adaptation (2017, background): “Some other procedures re-weight the data points in the source domain, to select source data that is similar to the target, when training a domain adaptive classifier, [9, 10, 19].”
Transfer Joint Matching for Unsupervised Domain Adaptation (2014, background): “Domain adaptation has proven to be promising in image classification [25, 13] and tagging [23, 26], object recognition [15, 2, 7, 10], and feature learning [14, 12, 22] In cross-domain problems, the source and target data are usually sampled from different probability distributions.”
Deep Transfer Network: Unsupervised Domain Adaptation (2015, background): “Instance reweighting [3, 4] and subspace learning [12, 11, 19, 27, 24] are two typical learning strategies for domain adaptation [22].”
Cross-dataset learning and person-specific normalisation for automatic Action Unit detection (2015, background): “It is interesting to note that the accurate recognition of certain AUs (2, 4, 9, 15, 17, 20) seems to require knowing the neutral facial expression whilst others can be recognised quite reliably without accounting for it (6, 7, 10, 12).”
Multisource Transfer Learning for Cross-Subject EEG Emotion Recognition (2020, background): “1) Instance transfer, which uses reweighted samples from the source to help the model training in the target [23], [24].”
AutoDIAL: Automatic Domain Alignment Layers (2017, background): “[4] formalized the two tasks of sample selection and classifier learning within a single optimization problem.”
Learning to Transfer: Transferring Latent Task Structures and Its Application to Person-Specific Facial Action Unit Detection (2015, background): “For example, [4] proposed a re-weighting of the source (training) distribution to match that of the domain (test) distribution.”
Deep Visual Domain Adaptation: A Survey (2018, background): “The common algorithms for shallow DA can mainly be categorized into two classes: instance-based DA [5, 12] and feature-based DA [30, 66, 23, 65].”
Adaptive SVM+: Learning with Privileged Information for Domain Adaptation (2017, methodology): “It has been applied with great success in applications ranging from cross-domain setups [9, 11, 38], to facial action unit detection in transductive learning setup [7], to deep neural networks [12, 39].”
Cross-Modal Supervision for Learning Active Speaker Detection in Video (2016, methodology): “Person-speciﬁc models There has been some work on person speciﬁc facial expression recognition and transferring generic to speciﬁc models for improving classiﬁcation performance [21,22,23].”
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition (2015, methodology): “[24] proposed a method that can be used in conjunction with available discriminative classifiers such as SVM.”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “s is possible, shallow DA methods described in the Section 3 can be applied to the problem. Hence, DA solutions such as Adaptive SVM [258], DT-SVM[68], Adaptive MKL [69] or Selective Transfer Machine [44], were applied to video concept detection [259], video event recognition [69], activity recognit”
Exploiting Low-Rank Structure from Latent Domains for Domain Generalization (2014, methodology): “We first investigate the state-of-the-art unsupervised domain adaptation methods, including Kernel Mean Matching (KMM) [21], Sampling Geodesic Flow (SGF) [17], Geodesic Flow Kernel (GFK) [16], Selective Transfer Machine(STM) [8], Domain Invariant Projection (DIP) [2], and Subspace Alignment (SA) [14”
A Unified Framework for Multimodal Domain Adaptation (2018, methodology): “There have been a bunch of methods for learning discriminative models that reduces the distribution mismatch between different domains [10, 19, 43, 44].”
Automatic Analysis of Facial Actions: A Survey (2019, methodology): “Building personalised models using online and transfer learning methodologies ([33], [29]) is the way forward in our opinion.”
Harnessing noisy Web images for deep representation (2017, background): “The second onstraint ensures the measurement α( x ) P ( x ) is close to a probaility distribution (for further details see Chu et al. (2013) ).”