The Painful Face – Pain Expression Recognition Using Active Appearance Models

By A. Ashraf, Simon Lucey, Jeffrey F. Cohn, Tsuhan Chen, Zara Ambadar, K. Prkachin, Patricia E. Solomon. Image and Vision Computing, 2009

373 citations recorded by the source corpus.

Pain is typically assessed by patient self-report. Self-reported pain, however, is difficult to interpret and may be impaired or in some circumstances (i.e., young children and the severely ill) not even possible. To circumvent these problems behavioral scientists have identified reliable and valid facial indicators of pain. Hitherto, these methods have required manual measurement by highly skilled human observers. In this paper we explore an approach for automatically recognizing acute pain without the need for human observers. Specifically, our study was restricted to automatically detecting pain in adult patients with rotator cuff injuries. The system employed video input of the patients as they moved their affected and unaffected shoulder. Two types of ground truth were considered. Sequence-level ground truth consisted of Likert-type ratings by skilled observers. Frame-level ground truth was calculated from presence/absence and intensity of facial actions previously associated with pain. Active appearance models (AAM) were used to decouple shape and appearance in the digitized face images. Support vector machines (SVM) were compared for several representations from the AAM and of ground truth of varying granularity. We explored two questions pertinent to the construction, design and development of automatic pain detection systems. First, at what level (i.e., sequence- or frame-level) should datasets be labeled in order to obtain satisfactory automatic pain detection performance? Second, how important is it, at both levels of labeling, that we non-rigidly register the face?

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/7e7ae434f75246d7d9b3fd649977e29194da0e1b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Categorical and dimensional affect analysis in continuous input: Current trends and future directions (2013, background): “The figure is courtesy of [1].”
Face alignment by coarse-to-fine shape searching (2015, background): “face verification and recognition [11], expression recognition [2], or facial attributes analysis [16].”
Affective Body Expression Perception and Recognition: A Survey (2013, background): “The research trend is now on naturally occurring affective expressions [159], [160] (yet they are still focused mainly on facial expressions).”
Fully Automatic Recognition of the Temporal Phases of Facial Actions (2012, neutral): “[17], pain [18], [19], and fatigue [20].”
Clinical Decision Support and Closed‐Loop Control for Intensive Care Unit Sedation (2013, neutral): “quantify agitation in sedated ICU patients [66–70].”
Detecting depression from facial actions and vocal prosody (2009, background): “Ashraf and Lucey et al. [ 15 ] detected pain on a frame-by-frame basis in participants with rotator cuff injuries.”
Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications (2016, background): “plex mental states [223], detection of fatigue [224], frustration [44], pain [185], [186], [225], severity of depression [53] and psychological distress [226], and including AFER capabilities”
Facial Expression Analysis (2011, background): “More recent work has attempted to detect expressions of pain [4, 79, 70], drowsiness, adult attachment [132], and indices of psychiatric disorder [27, 60].”
The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression (2010, background): “In previous work [3], it was shown by removing the rigid shape variation, poor performance was gained.”