Prosody-based automatic detection of annoyance and frustration in human-computer dialog

By J. Ang, Rajdip Dhillon, A. Krupski, Elizabeth Shriberg, Andreas Stolcke. 2002

439 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Social Psychology

Citations in the corpus

Generative Spoken Dialogue Language Modeling (2022, background): “However, all the progress in this research work centered around text dialogues leaving out non-lexical information (Schuller et al., 2013; Ang et al., 2002) in human-human dialogues, e.”
The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data (2007, neutral): “[1], [2], [3], [6], [10], [12]).”
Automatic detection of learner’s affect from conversational cues (2008, methodology): “scores among the emotions classified F(5,145) = 638.”
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications (2010, background): “Another approach involves a combination of acoustic-prosodic, lexical, and discourse features for affect detection [84], [168], [169]; however, the research incorporating this approach is too sparse to warrant a meaningful meta-analysis.”
Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments (2010, result): “Although the kappa score for the states was lower than the kappa for usage observations, it is on par with kappas reported by other researchers who have assessed the reliability of detecting naturally occurring emotional expressions (Ang et al., 2002; Grimm et. al., 2006; Litman & Forbes-Riley, 2004”
Toward detecting emotions in spoken dialogs (2005, methodology): “In [ 17 ], repetition or correction information was used for the discourse information.”
Challenges in real-life emotion annotation and machine learning based detection (2005, neutral): “…emotional behaviors or attitudes such as stressed vs. non-stressed speech (Fernandez & Picard, 2003; Narayanan, 2002; Petrushin, 1999), frustrated/annoyed vs. neutral/amused attitudes (Ang et al., 2002), or anger, motherese, emphatic and neutral (Steidl, Levit, Batliner, Nöth, & Niemann, 2005).”
Paralinguistics in speech and language - State-of-the-art and the challenge (2013, neutral): “…et al., 2008; Litman et al., 2009), deception (Enos et al., 2007; Bénézech, 2007), politeness (Nadeu and Prieto, 2011; Yildirim et al., 2005, 2011), frustration (Ang et al., 2002; Arunachalam et al., 2001; Lee et al., 2001), sarcasm (Rankin et al., 2009; Tepperman et al., 2006), pain (Belin et…”
EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks (2020, background): “The first one is based on nonphysiological signals, such as facial expression images [2], [3], [4], [5], [6], body gesture [7], and voice signal [8].”
Survey on speech emotion recognition: Features, classification schemes, and databases (2011, background): “Discourse information has been combined with acoustic correlates in order to improve the recognition performance of emotion recognition systems [8,3].”
Automatic speech recognition and speech variability: A review (2007, background): “Studies on speaker emotions is a fairly recent, emerging field and most of today’s literature that remotely deals with emotions in speech recognition is concentrated on attempting to classify a ‘‘stressed’’ or ‘‘frustrated’’ speech signal into its correct emotion category (Ang et al., 2002).”
Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features (2010, background): “…can be considered to be a separate modality from lexical features derived by transcribing the speech. involves a combination of acoustic-prosodic, lexical, and discourse features for affect detection (Ang et al. 2002; Lee and Narayanan 2005; Liscombe et al. 2005; Litman and Forbes-Riley 2004).”
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 (2012, background): “In the cases examined, items were phonemes, syllables, letters or words (Ang et al. 2002; Lee et al. 2002; Devillers et al. 2003).”
Predicting Student Emotions in Computer-Human Tutoring Dialogues (2004, methodology): “A number of researchers have accommodated for this low agreement by exploring ways of achieving consensus between disagreed annotations, to yield 100% agreement (e.g (Ang et al., 2002; Devillers et al., 2003)).”
EEG-based emotion recognition during watching movies (2011, background): “Ang and colleagues did emotion recognition based on prosody [ 3 ].”
Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge (2011, methodology): “Human–machine call-centre interaction (Ang et al., 2002; Batliner et al., 2004) is representative for telephone conversations in real-life settings.”
Enriching speech recognition with automatic detection of sentence boundaries and disfluencies (2006, background): “A discourse marker (DM) is a word or phrase that functions primarily as a structuring unit of spoken language, for example, actually, now, anyway, see, basically, so, I mean, well, let's see, you know, like.”
Crowd Truth: Harnessing disagreement in crowdsourcing a relation extraction gold standard (2013, background): “[2] and subsequent work in emotion [10], disagreement is used as a trigger for consensus-based annotation.”
Emotional speech recognition: Resources, features, and methods (2006, background): “Keywords: Emotions; Emotional speech data collections; Emotional speech classification; Stress; Interfaces; Acoustic features”