The Phasebook: Building Complex Masks via Discrete Representations for Source Separation
By Jonathan Le Roux, Gordon Wichern, Shinji Watanabe, Andy M. Sarroff, John R. Hershey. 2019
14 citations recorded by the source corpus.
Deep learning based speech enhancement and source separation systems have recently reached unprecedented levels of quality, to the point that performance is reaching a new ceiling. Most systems rely on estimating the magnitude of a target source, either directly or by computing a real-valued mask to be applied to a time-frequency representation of the mixture signal. A limiting factor in such approaches is a lack of phase estimation: the phase of the mixture is most often used when reconstructing the estimated time-domain signal. We propose to estimate phase using "phasebook", a new type of layer based on a discrete representation of the phase difference between the mixture and the target. We also introduce "combook", a similar type of layer that directly estimates a complex mask. We present various training and inference schemes involving these representations, and explain in particular how to include them in an end-to-end learning framework. We also present an oracle study to assess upper bounds on performance for various types of masks using discrete phase representations. We evaluate the proposed methods on the wsj0-2mix dataset, a well-studied corpus for single-channel speaker-independent speaker separation, matching the performance of state-of-the-art mask-based approaches without requiring additional phase reconstruction steps.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Yunpeng Li
- Marco Tagliasacchi
- Dominik Roblek
- Karolis Misiunas
- Zhihao Du
- Neil Zeghidour
- Pavel Andreev
- Zalán Borsos
Citations in the corpus
- SEANet: A Multi-modal Speech Enhancement Network (2020, background): “Speech denoising can also be seen as a special case of source separation, in which one of the sources represents the speech signal of interest [14, 15, 16, 17].”
- Wavesplit: End-to-End Speech Separation by Speaker Clustering (2020, unknown): “Two main families of algorithms have been proposed: frequency masking approaches (Hershey et al., 2016; Le Roux et al., 2019a) and time-domain approaches (Luo & Mesgarani, 2019; Luo et al., 2019).”