Low-Latency Real-Time Meeting Recognition and Understanding Using Distant Microphones and Omni-Directional Camera

By Takaaki Hori, Shoko Araki, Takuya Yoshioka, Masakiyo Fujimoto, Shinji Watanabe, T. Oba, Atsunori Ogawa, K. Otsuka, Dan Mikami, Keisuke Kinoshita and 3 more. 2012

88 citations recorded by the source corpus.

This paper presents our real-time meeting analyzer for monitoring conversations in an ongoing group meeting. The goal of the system is to recognize automatically “who is speaking what” in an online manner for meeting assistance. Our system continuously captures the utterances and face poses of each speaker using a microphone array and an omni-directional camera positioned at the center of the meeting table. Through a series of advanced audio processing operations, an overlapping speech signal is enhanced and the components are separated into individual speaker's channels. Then the utterances are sequentially transcribed by our speech recognizer with low latency. In parallel with speech recognition, the activity of each participant (e.g., speaking, laughing, watching someone) and the circumstances of the meeting (e.g., topic, activeness, casualness) are detected and displayed on a browser together with the transcripts. In this paper, we describe our techniques and our attempt to achieve the low-latency monitoring of meetings, and we show our experimental results for real-time meeting transcription.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Speech & Audio

In Signal Processing

Citations in the corpus

Generalization of Multi-Channel Linear Prediction Methods for Blind MIMO Impulse Response Shortening (2012, unknown): “(In addition to these experiments, we confirmed that the GWPE algorithm improves the performance of meeting speech recognition as detailed in [21].”
Hybrid acoustic models for distant and multichannel large vocabulary speech recognition (2013, methodology): “[10] describe a system which applies a dereverberation algorithm to the multichannel audio, followed by a source separation approach comprising a speaker diarisation component based on clustered direction-of-arrival estimates, which is then used to direct a delay-sum beamformer.”
Continuous Speech Separation: Dataset and Analysis (2020, background): “One approach could be to keep generating an enhanced signal in a continuous fashion for each involved speaker as in [26, 27], which requires online speaker diarization.”
Noise Model Transfer: Novel Approach to Robustness Against Nonstationary Noise (2013, methodology): “The enhanced feature vectors are finally fed into the back-end decoder after being converted to the corresponding mel-frequency cepstral coefficients (MFCCs), delta, and delta-delta cepstra.”