Lip Reading in the Wild

By Joon Son Chung, Andrew Zisserman. 2016

813 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

VoxCeleb: A Large-Scale Speaker Identification Dataset (2017, background): “This stage is closely related to the tracking pipeline of [34, 35], but optimised to reduce run-time given the very large number of videos to process.”
The Conversation: Deep Audio-Visual Speech Enhancement (2018, methodology): “The enhancement method proposed here is complementary to lip reading [31, 32, 33], which has also been shown to improve ASR performance in noisy environments [34, 35].”
Deep Audio-visual Learning: A Survey (2020, background): “Thus, researchers have tried to collect real-world videos from TV interviews, talks and movies and released several real-world datasets, including LRW, LRW variants [129, 59, 130] , Voxceleb and its variants [127, 128] , AVA-ActiveSpeaker [131] and AVSpeech [7] .”
Large-Scale Visual Speech Recognition (2018, methodology): “…these limitations, but most works still focus on single-word classiﬁcation, either by learning visual-only representations (Hinton et al., 2012; Chung & Zisserman, 2016a; Wand et al., 2016; Stafylakis & Tzimiropoulos, 2017; Wand & Schmidhuber, 2017), multimodal audio-visual representations…”
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild (2020, methodology): “t speaker. Another limitation of existing works is in terms of the vocabulary. Several works [5, 26, 28] train on datasets with a limited set of words such as GRID [10] (56 words), TIMIT [14] and LRW [8] (1000 words) which significantly hampers a model from learning the vast diversity of phoneme-vis”
Out of Time: Automated Lip Sync in the Wild (2016, methodology): “The visual part of the pipeline is based on the methods used by Chung and Zisserman [7], and we give a brief sketch of the method here.”
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (2021, methodology): “We leverage two in-the-wild audio-visual datasets which are popularly used in a great number of previous studies, VoxCeleb2 [15] and LRW [16].”
Learning Individual Speaking Styles for Accurate Lip to Speech Synthesis (2020, methodology): “Our model outperforms the baseline method proposed in [6], without any text-level supervision.”
Lip Reading in Profile (2017, methodology): “As shown in [10], the visual stream of this network generates excellent features for the task of lip reading – on the LRW [9] and OuluVS2 [2] datasets, single-layer classifiers trained on the SyncNet features have outperformed networks trained end-to-end on the task.”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “One of the first AV speech in-the-wild databases is LRW [40].”
Lip Reading Sentences in the Wild (2016, methodology): “The recent paper of Chung and Zisserman [9] tackles the small-lexicon problem by using faces in television broadcasts to assemble a dataset for 500 words.”
End-to-End Audiovisual Speech Recognition (2018, methodology): “For the purposes of this study we use the Lip Reading in the Wild (LRW) database [19] which is the largest publicly available lipreading dataset in the wild.”
Lipreading Using Temporal Convolutional Networks (2020, methodology): “dure to improve the generalization capabilities of the trained model when applied to sequences of varying length1. 2. DATABASES For the purposes of this study we use the Lip Reading in the Wild (LRW) [21] and LRW1000 [18] databases which are the largest publicly available lipreading datasets in Engl”
Perfect Match: Improved Cross-modal Embeddings for Audio-visual Synchronisation (2018, methodology): “We train and evaluate the models on the Lip Reading in the Wild (LRW) [14] dataset, which consists of wordlevel speech and video segments extracted from the British television.”
Combining Residual Networks with LSTMs for Lipreading (2017, methodology): “The best baseline result published in [4] is the multi-tower VGG-M.”
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement (2022, background): “The model demonstrate strong re-sults on two constrained datasets (LRW [8] and GRID [11]) which contain isolated words and ﬁxed-length sentences with limited word choices, respectively.”
Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset (2021, background): “On the other hand, most in-the-wild datasets do not focus on the task of talking face generation, e.g., Voxceleb[26, 8] is built for speaker identiﬁcation and LRW[9] is built for word recognition, so they do not pay attention to the video resolution.”
You said that? (2017, methodology): “We train the Speech2Vid model on videos from the VoxCeleb [19] and LRW [4] datasets.”