The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English
By Steven R. Livingstone, Frank A. Russo. 2018
2,268 citations recorded by the source corpus.
The RAVDESS is a validated multimodal database of emotional speech and song. The database is gender balanced consisting of 24 professional actors, vocalizing lexically-matched statements in a neutral North American accent. Speech includes calm, happy, sad, angry, fearful, surprise, and disgust expressions, and song contains calm, happy, sad, angry, and fearful emotions. Each expression is produced at two levels of emotional intensity, with an additional neutral expression. All conditions are available in face-and-voice, face-only, and voice-only formats. The set of 7356 recordings were each rated 10 times on emotional validity, intensity, and genuineness. Ratings were provided by 247 individuals who were characteristic of untrained research participants from North America. A further set of 72 participants provided test-retest data. High levels of emotional validity and test-retest intrarater reliability were reported. Corrected accuracy and composite "goodness" measures are presented to assist researchers in the selection of stimuli. All recordings are made freely available under a Creative Commons license and can be downloaded at https://doi.org/10.5281/zenodo.1188976.
Subfields: Experimental and Cognitive Psychology, Multimodal Learning, Evaluation & Benchmarks, Speech & Audio
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Experimental and Cognitive Psychology
- Soonil Kwon
- Muhammad Sajjad
- .. Mustaqeem
- Mustaqeem
- Ferran Hueto
- Leonardo Pepino
- Ran He
- Luciana Ferrer
In Multimodal Learning
In Evaluation & Benchmarks
- Ferran Hueto
- Leonardo Pepino
- Dilek Z. Hakkani-Tür
- Ran He
- Muhammad Sajjad
- Luciana Ferrer
- Dong Yu
- Jesper Højvang Jensen
In Speech & Audio
Citations in the corpus
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (2024, background): “2) RAVDESS [58], an emotional TTS dataset featuring 24 professional actors ( 12 female, 12 male) across 8 emotions (neutral, calm, happy, sad, angry, fearful, surprise, and disgust) in 2 emotional intensity (normal and strong).”
- The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems (2018, neutral): “The RAVDESS database emotional data for 24 different actors [16].”
- From Continuity to Editability: Inverting GANs with Consecutive Images (2021, methodology): “We first conduct our experiments on the RAVDESS dataset [12] with real videos.”
- Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings (2021, methodology): “The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [3] is a multi-modal database of emotional speech and song.”
- Clustering-Based Speech Emotion Recognition by Incorporating Learned Features and Deep BiLSTM (2020, methodology): “In this paper, we utilize the three public speech emotions datasets, the IEMOCAP [49] interactive emotional dyadic motion capture dataset, Emo-DB [50] berlin emotional dataset, and RAVDESS [51] Ryerson audio-visual dataset of emotional speech and songs.”
- COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings (2020, methodology): “a biomarker that detects this decline, we trained a Sentiment Speech classifier model to learn sentiment features on the RAVDESS speech dataset [32], which includes actors intonating in 8 emotional states: neutral, calm, happy, sad, angry, fearful, disgust, and surprised.”
- LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT (2023, methodology): “For the SER task, we collect corpora including MELD (Poria et al., 2018), IEMOCAP (Busso et al., 2008), RAVDESS (Livingstone & Russo, 2018), TESS (Pichora-Fuller & Dupuis, 2020), Crema-D (Cao et al., 2014), Emov-DB (Adigwe et al., 2018), and SAVEE (Jackson & Haq, 2014).”
- emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation (2023, positive): “We tested other speech emotion tasks, including song emotion recognition, emotion prediction in conversation, and sentiment analysis, on RAVDESS-Song (Livingstone and Russo, 2018), IEMOCAP and CMU-MOSI (Zadeh et al., 2016) & CMU-MOSEI (Zadeh et al., 2018).”
- Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systems (2021, background): “Amazon Alexa already has such a tone-detection feature deployed in production.6 There are academic data sets that map speech to emotion (Livingstone and Russo 2018), but academic benchmarks containing examples of intonation affecting the NLU output do not exist.”
- Deep Audio-visual Learning: A Survey (2020, methodology): “Classes Length* Year GRID [119] Lab 34 33,000 2006 Lombard Grid [120] Lab 54 54,000 2018 TCD TIMIT [121] Lab 62 2015 Vid TIMIT [122] Lab 43 2009 RAVDESS [123] Lab 24 2018 SEWA [124] Lab 180 2017 Speech OuluVS [125] Lab 20 1000 2009 OuluVS2 [126] Lab 52 3640 2016 Voxceleb [127] Wild 1,251 154,516 201”
- A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition (2019, methodology): “…were conducted on two standards benchmarked Interactive Emotional Dyadic Motion Capture (IEMOCAP) [12] and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [13] datasets to reveal the significance and e ffi ciency of the suggested model with other states of the art approaches.”
- An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, methodology): “RAVDESS [161] 2018 24 actors Continuous sentences 1920×1080 48 kHz Controlled environment – (12 males) and songs (2452 30 FPS Emotional speech and audio-visual clips) singing at two levels of intensity”
- One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing (2020, methodology): “We also use videos from the Ryerson audio-visual dataset [51] as well as a set of videos that we recorded with the permission from the subject ourselves.”
- Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset (2021, neutral): “5 51 RAVDESS[24] Lab 2018 1280×1024 24 7 8 MEAD[43] Lab 2020 1920×1080 60 40 159 Our HDTF Wild 2020 720P∼1080P 300+ 15.”