Emotion recognition using a hierarchical binary decision tree approach

By Chi-Chun Lee, Emily Mower Provost, Carlos Busso, Sungbok Lee, Shrikanth S. Narayanan. Speech Communication, 2011

452 citations recorded by the source corpus.

Automated emotion state tracking is a crucial element in the computational study of human communication behaviors. It is important to design robust and reliable emotion recognition systems that are suitable for real-world applications both to enhance analytical abilities to support human decision making and to design human-machine interfaces that facilitate efficient communication. We introduce a hierarchical computational structure to recognize emotions. The proposed structure maps an input speech utterance into one of the multiple emotion classes through subsequent layers of binary classifications. The key idea is that the levels in the tree are designed to solve the easiest classification tasks first, allowing us to mitigate error propagation. We evaluated the classification framework on two different emotional databases using acoustic features, the AIBO database and the USC IEMOCAP database. In the case of the AIBO database, we obtain a balanced recall on each of the individual emotion classes using this hierarchical structure. The performance measure of the average unweighted recall on the evaluation data set improves by 3.37% absolute (8.82% relative) over a Support Vector Machine baseline model. In the USC IEMOCAP database, we obtain an absolute improvement of 7.44% (14.58%) over a baseline Support Vector Machine modeling. The results demonstrate that the presented hierarchical approach is effective for classifying emotional utterances in multiple database contexts.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/fd84503ce8531f1b1757a3a8dd8e5189e49b6962

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings (2021, background): “In terms of modeling techniques, many different traditional approaches have been proposed, using hidden Markov models (HMMs) [7, 8], support vector machines (SVMs) [9, 10], decision trees [11, 12], and, more recently, deep neural networks (DNNs) [13, 14, 15].”
Databases, features and classifiers for speech emotion recognition: a review (2018, background): “Lee et al. (2011) computed zero crossing rate, root mean square energy, harmonics-to noise ratio and 12 mel frequency cepstral coefficients and their deltas for emotion analysis.”
Paralinguistics in speech and language - State-of-the-art and the challenge (2013, background): “…steps usually involve feature space optimisation by reduction and selection; this can be done ‘globally’, i.e., for all classes at a time, or ‘locally’, e.g., for sub-sets of classes in hierarchical classification (Lee et al., 2009, 2011b), or for different phonemes, etc. (Bitouk et al., 2011).”
Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge (2011, methodology): “On a diﬀerent basis, hierarchical approaches (Lee et al., 2009) to feature selection try to optimise the feature set not globally for all emotion classes but for groups of them, mainly couples.”
Multimodal Speech Emotion Recognition Using Audio and Text (2018, background): “Classical machine learning algorithms, such as hidden Markov models (HMMs), support vector machines (SVMs), and decision tree-based methods, have been employed in speech emotion recognition problems [12, 13, 14].”
Interactive Text-to-Speech via Semi-supervised Style Transfer Learning (2020, background): “Early approaches on emotion recognition mostly have been inspired by psychology studies [18, 19].”
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 (2012, background): “3 % (Lee et al. 2009).”
A Comprehensive Review of Speech Emotion Recognition Systems (2021, methodology): “Binary classiﬁcation is implemented in [74] instead of a single multiclass classiﬁer for emotion recognition.”
Speech emotion recognition using deep neural network and extreme learning machine (2014, methodology): “Some other classi-ﬁers, such as decision trees [13] and K-nearest neighbor (KNN) [14], have also been used in speech emotion recognition.”