Pyannote.Audio: Neural Building Blocks for Speaker Diarization

By Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, D. Fustes, Hadrien Titeux, Wassim Bouaziz, Marie-Philippe Gill. 2019

470 citations recorded by the source corpus.

We introduce pyannote.audio, an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it provides a set of trainable end-to-end neural building blocks that can be combined and jointly optimized to build speaker diarization pipelines. pyannote.audio also comes with pre-trained models covering a wide range of domains for voice activity detection, speaker change detection, overlapped speech detection, and speaker embedding – reaching state-of-the-art performance for most of them.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage (2021, background): “Since much of our dataset consists of conversational speech, developing an understanding of speaker changes, speaker overlap, and so on, perhaps by running a tool like pyannote [17], is also worthwhile.”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, methodology): “audio (Bredin et al., 2020) and adopt the nearest SD timestamps (by L1 distance to the original ones) instead for segmentation.”
Data Augmenting Contrastive Learning of Speech Representations in the Time Domain (2020, positive): “The recordings were cut into “utterance”-like segments using pyannote’s Voice Activity Detector [22].”
Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem (2021, methodology): “To further evaluate the purity of our generated samples, we use the overlapped speech length ratio calculated by the Pyannote Overlapped detection tool [32].”
Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts (2023, methodology): “Besides, since the speech in GigaSpeech and WenetSpeech does not have speaker identities and multiple speakers may appear in a speech clip, we process the datasets with an open-source automatic speaker diarization model 3 (Bredin et al., 2020; Bredin & Laurent, 2021).”
WhisperX: Time-Accurate Speech Transcription of Long-Form Audio (2023, unknown): “VAD Model pyannote [27] Onset threshold 0.”
Towards Unsupervised Learning of Speech Features in the Wild (2021, methodology): “In the ﬁrst two subsets, the non-speech parts were ﬁltered out using Voice Activity Detection (VAD) computed with pyan-note.audio [37].”
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition (2023, methodology): “For speaker diarization evaluation, Diarization Error Rate (DER) is reported on three datasets: (a) AMI Corpus [25], comprising Lapel and MixHeadset audio subsets from the partition set [26].”
SpeechBrain: A General-Purpose Speech Toolkit (2021, unknown): “CTC [24] Transducers [25] CTC+Attention [26] Shallow fusion [27] LibriSpeech [28] Common Voice [29] AISHELL [30] TIMIT [31] separation, pyannote [17] and sidekit [18] for speaker diarization, and s3prl [19] for self-supervised speech representations.”
End-to-end speaker segmentation for overlap-aware resegmentation (2021, methodology): “1 pretrained pipelines [16], dihard3 official baseline [9], and BUT’s VBx approach [15].”
Pyannote.audio 2.1 Speaker Diarization Pipeline: Principle, Benchmark, and Recipe (2023, background): “9% with the xvector implementation available in pyannote.audio [20], 12 .”