Powerset multi-class cross entropy loss for neural speaker diarization

By Alexis Plaquet, Hervé Bredin. 2023

236 citations recorded by the source corpus.

Since its introduction in 2019, the whole end-to-end neural diarization (EEND) line of work has been addressing speaker diarization as a frame-wise multi-label classification problem with permutation-invariant training. Despite EEND showing great promise, a few recent works took a step back and studied the possible combination of (local) supervised EEND diarization with (global) unsupervised clustering. Yet, these hybrid contributions did not question the original multi-label formulation. We propose to switch from multi-label (where any two speakers can be active at the same time) to powerset multi-class classification (where dedicated classes are assigned to pairs of overlapping speakers). Through extensive experiments on 9 different benchmarks, we show that this formulation leads to significantly better performance (mostly on overlapping speech) and robustness to domain mismatch, while eliminating the detection threshold hyperparameter, critical for the multi-label formulation.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Audiobox: Unified Audio Generation with Natural Language Prompts (2023, methodology): “We create the transcribed subset with the following pre-processing methods: For unsegmented multi-speaker conversational datasets information, we first segment our dataset using PyAnnote diarization toolkit (Plaquet and Bredin, 2023; Bredin, 2023) to create single speaker speech segments.”