Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals

By Christine L. Lisetti, Fatma Nasoz. EURASIP Journal on Advances in Signal Processing, 2004

477 citations recorded by the source corpus.

We discuss the strong relationship between affect and cognition and the importance of emotions in multimodal human computer interaction (HCI) and user modeling. We introduce the overall paradigm for our multimodal system that aims at recognizing its users' emotions and at responding to them accordingly depending upon the current context or application. We then describe the design of the emotion elicitation experiment we conducted by collecting, via wearable computers, physiological signals from the autonomic nervous system (galvanic skin response, heart rate, temperature) and mapping them to certain emotions (sadness, anger, fear, surprise, frustration, and amusement). We show the results of three different supervised learning algorithms that categorize these collected signals in terms of emotions, and generalize their learning to recognize emotions from new collections of signals. We finally discuss possible broader impact and potential applications of emotion recognition for multimodal intelligent systems.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/1d61c66da206f98780c6619814be9b22a1ebe2e5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty (2011, background): “In [6], six emotional states, elicited by film clips, were classified with an accuracy of 84%.”
Adaptive interfaces and agents (2002, background): “…transmit physiological data, such as electromyogram signals, the galvanic skin response, blood volume pressure, and the pattern of respiration (see Lisetti & Nasoz, 2004, for an overview); and (b) video cameras and microphones that transmit psychologically relevant information about the user,…”
Enabling Effective Programming and Flexible Management of Efficient Body Sensor Network Applications (2013, background): “In [15], a study on using BSN technology to recognize human emotions has been pre-”
ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors (2018, methodology): “Finally, the importance of using less-intrusive sensors in affective studies has been widely acknowledged [23], [25].”
Short-term emotion assessment in a recall paradigm (2009, methodology): “For emotions elicited by an event the best results obtained on more than three classes are those presented in (Lisetti and Nasoz, 2004) and (Wagner et al., 2005).”
Empirically building and evaluating a probabilistic model of user affect (2009, background): “When emotions are varied and rapidly changing, it is hard for the users to describe them by using post-treatment self-reports, as was done for instance in (Lisetti & Nasoz, 2004; Peter & Herbon, 2006) to label valence/arousal or the occurrence of one spefic emotion.”
Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals (2006, background): “In recent years interesting results have been obtained with the first category of signals ([7], [8]).”
Human Emotion Recognition: Review of Sensors and Methods (2020, background): “For example, in [184] introduced the overall paradigm for their multimodal system that aims at recognizing its users’ emotions and responding to them accordingly depending upon the current context or application.”
Fundamentals of physiological computing (2009, background): “Other research has elected to represent user state in terms of: distinct categories of emotion (Healey and Picard, 1997; Lisetti and Nasoz, 2004; Lisetti et al., 2003), twodimensional space of activation and valence (Kulic and Croft, 2005, 2006) and distinct emotional categories based upon a two-dim”
Multimodal Emotion Recognition in Response to Videos (2012, background): “[3] M. Soleymani, G. Chanel, J. J. M. Kierkels, and T. Pun,“ Affective Characterization of Movie Scenes Based on Content Analysis and Physiological Changes,”International Journal of Semantic Computing, vol. 3, no. 2, pp. 235-254, June 2009.”
Objective measures, sensors and computational techniques for stress recognition and classification: A survey (2012, methodology): “Emotions of anger, fear and frusration have been classified using GSR, HR and ST [54].”
DEAP: A Database for Emotion Analysis ;Using Physiological Signals (2012, background): “There has been a large number of published works in the domain of emotion recognition from physiological signals [16], [20], [21], [22], [23], [24].”
A Review of Emotion Recognition Using Physiological Signals (2018, methodology): “In the study of [53], subjects were required to watch a 45-min slide show while their galvanic skin response (GSR), heart rate (HR), and temperature were measured using Body Media Sense Wear Armband (Figure 8d).”
EEG-based classification of positive and negative affective states (2014, background): “Interestingly, these two studies used continuous stimuli such as music [28] and movies [36] to elicit”
DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses (2015, unknown): “Index Terms—Emotion recognition, user physiological responses, MEG, single-trial classification, affective computing Ç”