Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

By George Papandreou, Athanasios Katsamanis, Vassilis Pitsikalis, Petros Maragos. 2009

127 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Signal Processing

Citations in the corpus

Deep Learning: Methods and Applications (2014, methodology): “However, the same paper also shows that a model of [278] in which a sophisticated signal processing technique for extracting visual features, together with the uncertainty-compensation method developed originally from robust speech recognition [104], gives the best classification accuracy in the cro”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “However, the same paper also shows that a model of [194] in which a sophisticated signal processing technique for extracting visual features, together with the uncertainty-compensation method developed originally from robust speech recognition [195], gives the best classification accuracy in the cro”
Ava Active Speaker: An Audio-Visual Dataset for Active Speaker Detection (2019, unknown): “Conversational content in videos has received significant attention, with audio-only, visual-only and joint audio-visual modeling approaches [1, 2, 3, 4, 5, 6] used for applications in human-robot interaction and speech recognition.”
Large-Scale Visual Speech Recognition (2018, background): “While there is a large body of literature on automated lipreading, much of the early work focused on single-word classification and relied on substantial prior knowledge (Chu & Huang, 2000; Matthews et al., 2002; Pitsikalis et al., 2006; Lucey & Sridharan, 2006; Papandreou et al., 2007; Zhao et al.,”
Combining Residual Networks with LSTMs for Lipreading (2017, unknown): “Prior to the advent of deep learning ([10]) most of the work in lipreading was based on hand-engineered features, that were usually modeled by HMM-based pipeline, [11] [12] [13] [14] [15].”
Multimodal Deep Learning (2011, methodology): “1% † Visemic AAM[23] 83% †§ (b) CUAVE Video Table 1: Classification performance for visual speech classification on (a) AVLetters and (b) CUAVE.”
Multimodal learning with deep Boltzmann machines (2012, background): “08 Visemic AAM (Papandreou et al., 2009) 83 Table 5: Classification results on the CUAVE data set.”
LipNet: End-to-End Sentence-level Lipreading (2016, background): “…of frames to extract video features (e.g., optical ﬂow or movement detection), or other types of handcrafted vision pipelines (Matthews et al., 2002; Zhao et al., 2009; Gurban & Thiran, 2009; Papandreou et al., 2007; 2009; Pitsikalis et al., 2006; Lucey & Sridharan, 2006; Papandreou et al., 2009).”