A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition

By Yong-jin Liu, Jinkai Zhang, Wen-Jing Yan, Sujing Wang, Guoying Zhao, Xiaolan Fu. IEEE Transactions on Affective Computing, 2016

464 citations recorded by the source corpus.

Micro-expressions are brief facial movements characterized by short duration, involuntariness and low intensity. Recognition of spontaneous facial micro-expressions is a great challenge. In this paper, we propose a simple yet effective Main Directional Mean Optical-flow (MDMO) feature for micro-expression recognition. We apply a robust optical flow method on micro-expression video clips and partition the facial area into regions of interest (ROIs) based partially on action units. The MDMO is a ROI-based, normalized statistic feature that considers both local statistic motion information and its spatial location. One of the significant characteristics of MDMO is that its feature dimension is small. The length of a MDMO feature vector is 36 × 2 = 72, where 36 is the number of ROIs. Furthermore, to reduce the influence of noise due to head movements, we propose an optical-flow-driven method to align all frames of a micro-expression video clip. Finally, a SVM classifier with the proposed MDMO feature is adopted for micro-expression recognition. Experimental results on three spontaneous micro-expression databases, namely SMIC, CASME and CASME II, show that the MDMO can achieve better performance than two state-of-the-art baseline features, i.e., LBP-TOP and HOOF.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Urban Studies

View Paper: https://www.semanticscholar.org/paper/cc6e844fbc30fdbcb8163768ffe2886c4cb0283e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks (2020, background): “The first one is based on nonphysiological signals, such as facial expression images [2], [3], [4], [5], [6], body gesture [7], and voice signal [8].”
A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework (2019, background): “Obvious improvements are also observed in [21], [20], [22] as a result of introducing MAG.”
OFF-ApexNet on Micro-expression Recognition System (2018, background): “For instance, [39] proposed to construct a RoI-based feature vector using optical ﬂow to describe the local motion information and the spatial location.”
SAMM: A Spontaneous Micro-Facial Movement Dataset (2018, methodology): “To further expand on the testing results, we employ a method similarly used in [27] to leave each subject out for testing and use the rest of the data for training.”
A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, background): “When it comes to different mechanisms of facial expression [222], FER can be further divided into macro-FER and micro-FER [214,223,224].”
Video-Based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms (2021, methodology): “…50.0 Transfer Learning [121] 52.4 55.9 Multi-task mid-level feature learning [62] 54.2 55.0 KGSL [61] 56.9 48.6 Sparse MDMO [86] 60.0 52.9 MDMO [85] 65.7 50.0 DTSCNN [94] 65.9 69.2 TLCNN [96] 69.4 73.5 TOP [40] and LBP-MOP [45]), multi-task mid-level feature learning [62] can enhance the…”
Less is More: Micro-expression Recognition from Video using Apex Frame (2016, background): “Optical flow estimates the motion of objects between two images over time, which was found effective in a recently proposed micro-expression recognition system [21].”