A Review of Speaker Diarization: Recent Advances with Deep Learning

By Tae Jin Park, Naoyuki Kanda, Dimitrios Dimitriadis, Kyu J. Han, Shinji Watanabe, Shrikanth S. Narayanan. 2021

437 citations recorded by the source corpus.

Speaker diarization is a task to label audio or video recordings with classes that correspond to speaker identity, or in short, a task to identify"who spoke when". In the early years, speaker diarization algorithms were developed for speech recognition on multispeaker audio recordings to enable speaker adaptive processing. These algorithms also gained their own value as a standalone application over time to provide speaker-specific metainformation for downstream tasks such as audio retrieval. More recently, with the emergence of deep learning technology, which has driven revolutionary changes in research and practices across speech application domains, rapid advancements have been made for speaker diarization. In this paper, we review not only the historical development of speaker diarization technology but also the recent advancements in neural speaker diarization approaches. Furthermore, we discuss how speaker diarization systems have been integrated with speech recognition applications and how the recent surge of deep learning is leading the way of jointly modeling these two components to be complementary to each other. By considering such exciting technical trends, we believe that this paper is a valuable contribution to the community to provide a survey work by consolidating the recent developments with neural methods and thus facilitating further progress toward a more efficient speaker diarization.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[177] Tae Jin Park, Naoyuki Kanda, Dimitrios Dimitriadis, Kyu J Han, Shinji Watanabe, and Shrikanth Narayanan.”
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, background): “Speaker diarization is a task to recognize “who spoke when” from an input audio stream [22].”
Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors (2021, background): “This task is called speaker diarization, and in recent years there have been extensive efforts toward accurate speaker diarization in various domains [1].”
Pyannote.audio 2.1 Speaker Diarization Pipeline: Principle, Benchmark, and Recipe (2023, methodology): “Although spectral clustering [6] and variational Bayesian hidden Markov models [5] have been the prefered clustering techniques in recent speaker diarization literature [7], the proposed pipeline relies on a classical agglomerative hierarchical clustering with centroid linkage (also known as the UPG”
Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone (2021, methodology): “To handle this, one approach is to first apply speaker diarization [11, 12, 13] to detect the utterance boundaries for each speaker and then perform ASR for each utterance.”