Hidden Markov model-based speech emotion recognition

By Björn Schuller, Gerhard Rigoll, Manfred K. Lang. 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698), 2003

693 citations recorded by the source corpus.

In this contribution we introduce speech emotion recognition by use of continuous hidden Markov models. Two methods are propagated and compared throughout the paper. Within the first method a global statistics framework of an utterance is classified by Gaussian mixture models using derived features of the raw pitch and energy contour of the speech signal. A second method introduces increased temporal complexity applying continuous hidden Markov models considering several states using low-level instantaneous features instead of global statistics. The paper addresses the design of working recognition engines and results achieved with respect to the alluded alternatives. A speech corpus consisting of acted and spontaneous emotion samples in German and English language is described in detail. Both engines have been tested and trained using this equivalent speech corpus. Results in recognition of seven discrete emotions exceeded 86% recognition rate. As a basis of comparison the similar judgment of human deciders classifying the same corpus at 79.8% recognition rate was analyzed.

Subfields: Experimental and Cognitive Psychology, Pharmacy, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f106e3afbdd01022962d7a73a847869d0534a036

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Pharmacy

In Signal Processing

Citations in the corpus

Objective measures, sensors and computational techniques for stress recognition and classification: A survey (2012, methodology): “HMMs have been sed for modelling emotions from voice data as well [124].”
EmotionSense: a mobile phones based adaptive platform for experimental social psychology research (2010, methodology): “We plan to improve the emotion classifiers by optimizing the size of model and the PLP front-ends in order to obtain an optimal one for emotion recognition [27] and by connecting external sensors such Galvanic Skin Response device.”
Multimodal Speech Emotion Recognition Using Audio and Text (2018, background): “Classical machine learning algorithms, such as hidden Markov models (HMMs), support vector machines (SVMs), and decision tree-based methods, have been employed in speech emotion recognition problems [12, 13, 14].”
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 (2012, methodology): “Hidden Markov Models (HMMs) are considered to be a common classification/modeling technique for speech emotion recognition (Schuller et al. 2003; Pao et al. 2007a; Ijima et al. 2009; Yun and Yoo 2009; Yu 2008; Fu et al. 2008b; Nogueiras et al. 2001).”
Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge (2011, methodology): “…Models (GMM, i.e., continuous HMM with a single state that use weighted mixtures of Gaussian densities) (Neiberg et al., 2006; Steidl, 2009) and HMM (Nogueiras et al., 2001; Nwe et al., 2003; Schuller et al., 2003; Yu et al., 2004; Inanoglu and Caneel, 2005; Nose et al., 2007; Wagner et al., 2007).”
SPEECH EMOTION RECOGNITION USING SUPPORT VECTOR MACHINE (2010, background): “Furthermore, many researchers explored several classification methods, such as Neural Networks (NN) [5], Gaussian Mixture Model (GMM), Hidden Markov model (HMM) [6], Maximum Likelihood Bayesian classifier (MLC), Kernel Regression and K-nearest Neighbors (KNN) and Support vector machines (SVM) [7].”
Speech emotion recognition using convolutional and Recurrent Neural Networks (2016, background): “Traditional speech recognition methods are proposed in many research studies [3-5].”
Survey on speech emotion recognition: Features, classification schemes, and databases (2011, methodology): “Based on several studies [94,21,72,97,115,43,138,129], we can conclude that HMM is the most used classifier in emotion classification probably because it is widely used in almost all speech applications.”
Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture (2004, methodology): “In former works [7] we compared static and dynamic feature sets for the prosodic analysis.”
Speech emotion recognition using deep neural network and extreme learning machine (2014, methodology): “Some studies used generative models, such as Gaussian mixture models (GMMs) and Hidden Markov models (HMMs), to learn the distribution of these low-level features, and then use the Bayesian classiﬁer or the maximum likelihood principle for emotion recognition [7, 8].”
Sentiment Adaptive End-to-End Dialog Systems (2018, methodology): “Many studies in emotion recognition (Schuller et al., 2003; Nwe et al., 2003; Bertero et al., 2016) have used only acoustic features.”