CUAVE: A new audio-visual database for multimodal human-computer interface research

By E. Patterson, S. Gurbuz, Z. Tufekci, J. Gowdy. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002

360 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/dd70cd5c716b7c484f4a9833cf61453e1e70d387

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Kristen Grauman
2. Ruohan Gao
3. Zheng-Hua Tan
4. Daniel Michelsanti
5. Dong Yu
6. Yong Xu
7. Ruslan Salakhutdinov
8. Shi-Xiong Zhang

In Signal Processing

1. Dong Yu
2. Li Deng
3. Kristen Grauman
4. Jinyu Li
5. Ruohan Gao
6. Andrew W. Senior
7. Zheng-Hua Tan
8. Shinji Watanabe

Citations in the corpus

• Multimodal learning with deep Boltzmann machines (2012, methodology): “CUAVE (Patterson et al., 2002): This data set consists of 36 speakers speaking the digits 0 to 9.”
• Combining Residual Networks with LSTMs for Lipreading (2017, background): “Moreover, the number of target words is 500, which is an order of magnitude higher than other publicly available databases (GRID [24], CUAVE [26], a.o.).”
• Recent advances in the automatic recognition of audiovisual speech (2003, methodology): “CUAVE I/C-digits 36 US [28], [87], [135]”
• Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition (2009, methodology): “The proposed scheme for fusion by uncertainty compensation has been evaluated with audiovisual speech recognition experiments.”
• VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency (2021, methodology): “In Table 3a-3c, we evaluate on the Mandarin, TCD-TIMIT and CUAVE datasets using our speaker-independent model trained on VoxCeleb2 to test the cross-dataset generalization capability of our model.”
• Ava Active Speaker: An Audio-Visual Dataset for Active Speaker Detection (2019, methodology): “Early corpora were designed for speech (digit) recognition with high-resolution, frontal facing speaking faces, such as CUAVE [38] and AVTIMIT [25].”
• Deep Computation Model for Unsupervised Feature Learning on Big Data (2016, methodology): “Specifically, we use the features to classify the speech in the CUAVE dataset which is composed of 36 individuals saying the digits 0 to 9 [22].”
• An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “CUAVE [204] 2002 36 (19 males) Connected and isolated 720×480 Stereo 44 kHz Controlled environment [55] and 20 pairs digits (7,000 utterances) 29.”
• Learning to lip read words by watching videos (2018, neutral): “5% [17] CUAVE [18] Lab Digits I 10 36 83.”
• Multimodal Deep Learning (2011, neutral): “CUAVE [15].”