Acoustic Beamforming for Speaker Diarization of Meetings

By Xavier Anguera Miró, Chuck Wooters, Javier Hernando. 2007

424 citations recorded by the source corpus.

When performing speaker diarization on recordings from meetings, multiple microphones of different qualities are usually available and distributed around the meeting room. Although several approaches have been proposed in recent years to take advantage of multiple microphones, they are either too computationally expensive and not easily scalable or they cannot outperform the simpler case of using the best single microphone. In this paper, the use of classic acoustic beamforming techniques is proposed together with several novel algorithms to create a complete frontend for speaker diarization in the meeting room domain. New techniques we are presenting include blind reference-channel selection, two-step time delay of arrival (TDOA) Viterbi postprocessing, and a dynamic output signal weighting algorithm, together with using such TDOA values in the diarization to complement the acoustic information. Tests on speaker diarization show a 25% relative improvement on the test set compared to using a single most centrally located microphone. Additional experimental results show improvements using these techniques in a speech recognition task.

Subfields: Computational Mechanics, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Speech & Audio

In Signal Processing

Citations in the corpus

CHiME-6 Challenge: Tackling Multispeaker Speech Recognition for Unsegmented Recordings (2020, positive): “Therefore, by comparing Tracks 1 and 2 with BeamformIt, we can observe that the main degradation (around 15% absolute) comes from speaker diarization.”
An analysis of environment, microphone and data simulation mismatches in robust speech recognition (2017, positive): “Fixed beamformers such as DS and its variant known as BeamformIt (Anguera et al., 2007) which was used in many challenge submissions do not suffer from this issue due to the fact that their spatial response decays slowly in the neighborhood of the estimated speaker direction.”
The third 'CHiME' speech separation and recognition challenge: Analysis and outcomes (2017, methodology): “Of these, several reported that the freely available BeamformIt tool developed by Anguera et al. (2007) worked very eﬀectively.”
A Study of Learning Based Beamforming Methods for Speech Recognition (2016, methodology): “Comparing with the weighted delay-and-sum implemented in BeamformIt [28] (row 1), the neural networks perform slightly worse in overall, and LSTM performs slightly better than DNN. Row 4 and 5 show the results of CE training in which the neural networks are re- fined using the ASR cross entropy cost”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, background): “The new approaches resulting from these advancements include, but not limited to, beamforming [28], information bottleneck clustering”
Acoustic Modeling for Distant Multi-talker Speech Recognition with Single- and Multi-channel Branches (2019, methodology): “The best results were obtained when we fed the “ArrayRaw” and “BeamformIt” data into the 1-ch input branch in training, and BeamformIt was applied to the 1-ch input branch in decoding.”
Hybrid acoustic models for distant and multichannel large vocabulary speech recognition (2013, methodology): “Both of these systems process the microphone array signals using a noise-reducing Wiener filter on each channel, followed by delay-sum beamforming where the time delays of arrival are estimated using generalized cross-correlation with phase transform (GCC-PHAT) [5] and smoothed using a two-stage Vit”
Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone (2021, methodology): “While our main focus in this paper is SDM-based recognition, we also evaluated the fine-tuned single-talker ASR model for the MDM setting using BeamformIt beamformer [42].”
Speaker Diarization: A Review of Recent Research (2010, methodology): “Since the NIST RT‘05 evaluation, the most common approach to multi-channel speaker diarization involves acou stic beamforming as initially proposed in [41] and described in detail in [42].”
Neural network based spectral mask estimation for acoustic beamforming (2016, methodology): “BeamformIt! employs a Delay-and-Sum beamformer where the DoA estimate is obtained from GCC-PHAT [23] and post-processed to avoid instabilities.”
The Second DIHARD Diarization Challenge: Dataset, task, and baselines (2019, methodology): “For the multichannel tracks, we use weighted delay-and-sum beamforming as implemented in BeamformIt [31].”
Improved MVDR Beamforming Using Single-Channel Mask Prediction Networks (2016, methodology): “These time delay candidates are calculated for each segment of the signal and reconciled across segments using a Viterbi search [21].”
The Automatic Speech recogition In Reverberant Environments (ASpIRE) challenge (2015, methodology): “System 18 was comparable to system 4, except that the signal processing pipeline used a beam-forming algorithm [35], and semi-supervised training was based on six times the evaluation data used in the single microphone condition.”
The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines (2018, positive): “CHiME-5 uses a weighted delay-and-sum beamformer (BeamformIt [32]) as a default multichannel speech enhancement approach, similar to the CHiME-4 recipe [11].”
The USTC-iFlytek systems for CHiME-5 Challenge (2018, methodology): “The front-end uses a weighted delay-and-sum beamformer by BeamformIt toolkit [15] as a default multichannel speech enhancement approach.”