A Brief Review of Facial Emotion Recognition Based on Visual Information

By ByoungChul Ko. Italian National Conference on Sensors, 2018

694 citations recorded by the source corpus.

Facial emotion recognition (FER) is an important topic in the fields of computer vision and artificial intelligence owing to its significant academic and commercial potential. Although FER can be conducted using multiple sensors, this review focuses on studies that exclusively use facial images, because visual expressions are one of the main information channels in interpersonal communication. This paper provides a brief review of researches in the field of FER conducted over the past decades. First, conventional FER approaches are described along with a summary of the representative categories of FER systems and their main algorithms. Deep-learning-based FER approaches using deep networks enabling “end-to-end” learning are then presented. This review also focuses on an up-to-date hybrid deep-learning approach combining a convolutional neural network (CNN) for the spatial features of an individual frame and long short-term memory (LSTM) for temporal features of consecutive frames. In the later part of this paper, a brief review of publicly available evaluation metrics is given, and a comparison with benchmark results, which are a standard for a quantitative comparison of FER researches, is described. This review can serve as a brief guidebook to newcomers in the field of FER, providing basic knowledge and a general understanding of the latest state-of-the-art studies, as well as to experienced researchers looking for productive directions for future work.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/1e7ae86a78a9b4860aa720fb0fd0bdc199b092c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Human-Computer Interaction

Citations in the corpus

Artificial intelligence in the creative industries: a review (2020, background): “eyebrows, the tip of nose, the corners of mouth) [282], and both audio and video [283].”
Privileged Information for Modeling Affect In The Wild (2021, methodology): “Popular visual descriptors, such as SIFT [12], HOG [13] and LBP [14] have been widely used to represent human facial patterns via a set of features used as inputs to machine learning models for emotion recognition [15], [16].”
A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, methodology): “For visual modality, most studies surveyed FER [38,39,41,44] and FMEA [40].”
Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition (2021, background): “Traditional emotion recognition systems are built with nonphysiological signals [4], [5].”
EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications (2020, background): “EEG signals for emotion recognition and classification in numerous recent studies [111], [112], [113], [114].”