Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech

By Ginevra Castellano, Loïc Kessous, George Caridakis. Affect and Emotion in Human-Computer Interaction, 2008

317 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Review and Meta-Analysis of Multimodal Affect Detection Systems (2015, methodology): “…segment-level analyses with a temporal resolution in seconds were preferred over frame-level analyses with a temporal resolution in milliseconds because affective phenomena operate across a coarser time span ranging from a few seconds to tens of seconds [D’Mello and Graesser 2011; Rosenberg 1998].”
Developing crossmodal expression recognition based on a deep neural model (2016, methodology): “This was demonstrated in the computational system of Castellano, Kessous, and Caridakis (2008), where they process facial expression, body posture, and speech, extracting a series of features from each modality and combining them into one feature vector.”
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos (2018, background): “Hence, many works used visual and audio features together for detecting affect (Busso et al., 2004; Castellano et al., 2008; Ranganathan et al., 2016).”
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications (2010, background): “acted expressions [171], [201], converging research points to”
Survey on Emotional Body Gesture Recognition (2018, background): “the hand’s barycenter, which was extended in [105], adopting multiple modalities (face, body gesture, speech), and in [93], considering 3D features instead.”