How to find trouble in communication

By Anton Batliner, Kerstin Fischer, R. Huber, J. Spilker, Elmar Nöth. Speech Communication, 2003

326 citations recorded by the source corpus.

Automatic dialogue systems used, for instance, in call centers, should be able to determine in a critical phase of the dialogue-indicated by the customers vocal expression of anger/irritation-when it is better to pass over to a human operator. At a first glance, this does not seem to be a complicated task: It is reported in the literature that emotions can be told apart quite reliably on the basis of prosodic features. However, these results are achieved most of the time in a laboratory setting, with experienced speakers (actors), and with elicited, controlled speech. We compare classification results obtained with the same feature set for elicited speech and for a Wizard-of-Oz scenario, where users believe that they are really communicating with an automatic dialogue system. It turns out that the closer we get to a realistic scenario, the less reliable is prosody as an indicator of the speakers' emotional state. As a consequence, we propose to change the target such that we cease looking for traces of particular emotions in the users' speech, but instead look for indicators of TROUBLE IN COMMUNICATION. For this reason, we propose the module Monitoring of User State [especially of] Emotion (MOUSE) in which a prosodic classifier is combined with other knowledge sources, such as conversationally peculiar linguistic behavior, for example, the use of repetitions. For this module, preliminary experimental results are reported showing a more adequate modelling of TROUBLE IN COMMUNICATION.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Language and Linguistics

View Paper: https://www.semanticscholar.org/paper/e6ba4cde252bb2e988c325369716f29f8083d46a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Language and Linguistics

R. Cowie

Citations in the corpus

Abandoning emotion classes - towards continuous emotion recognition with modelling of long-range dependencies (2008, unknown): “[1].”
Challenges in real-life emotion annotation and machine learning based detection (2005, unknown): “…combine the information of different natures, paralinguistic features (prosodic, spectral, disfluences, etc.) with linguistic features (lexical, dialogic), to improve emotion detection or prediction is also a research challenge (Batliner et al., 2003; Forbes-Riley & Litman, 2004; Narayanan, 2002).”
Describing the emotional states that are expressed in speech (2003, methodology): “To illustrate it, consider two proposals, one from Roseman (1991) and the other from Ortony et al. (1988). Both include two distinctions that can be regarded as basic––whether the key elements of the situation are positively or negatively evaluated in themselves, and whether or not they help the age”
Predicting Student Emotions in Computer-Human Tutoring Dialogues (2004, background): “While acoustic-prosodic features address how something is said, lexical features representing what is said have also been shown to be useful for predicting emotion in spontaneous dialogues (Lee et al., 2002; Ang et al., 2002; Batliner et al., 2003; Devillers et al., 2003; Shafran et al., 2003).”
Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge (2011, background): “…detection in a driving simulation (Fernandez and Picard, 2003), tutoring dialogues (Litman and Forbes, 2003; Liscombe, 2005), information systems (Batliner et al., 2003b), or human–robot communication (Batliner et al., 2008b); a still emerging ﬁeld is interaction with virtual agents (Schro¨der…”
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 (2012, background): “In an early research Batliner et al. (2003) suggested that apart from prosodic features in the speech signal, syntactic and behavioral hints like repetitions in a dialogue and partof-speech features, should be taken into consideration. According to this article, repetitions and reformulations (the u”
How emotion is made and measured (2007, background): “As a more recent example of this historical disjuncture between emotions found in the laboratory and those ''in the wild,'' Batliner et al. (2003) review the performance of automatic call-in systems that use experimental pre-tests of the software's ability to detect frustration in a caller's voice.”
ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors (2018, methodology): “modalities employed for AR include audio [14], [15], [16], visual [17], [18], [19] and audio-visual [20], [21], [22].”
Emotion Recognition (2009, background): “As found in the literature [1, 4], emotion in spoken dialogs is not limited to humanhuman interaction but is also found in human-computer interaction.”
The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data (2007, background): “The move was undoubtedly related to the recognition that systems trained on acted stereotypical data do not transfer to more everyday situations [5].”
Improving Automatic Emotion Recognition from Speech via Gender Differentiaion (2006, methodology): “…the current state-of-the-art in speech emotion recognition – a too complex task, we merged them into a 4-class problem applying a scheme suggested in (Batliner et al., 2003b): neutral and unidentiﬁable utterances to neutral , strong and weak joy and strong and weak surprise into joy , strong and…”