Speaker-Independent Speech Separation With Deep Attractor Network

By Yi Luo, Zhuo Chen, Nima Mesgarani. 2017

266 citations recorded by the source corpus.

Despite the recent success of deep learning for many speech processing tasks, single-microphone, speaker-independent speech separation remains challenging for two main reasons. The first reason is the arbitrary order of the target and masker speakers in the mixture (permutation problem), and the second is the unknown number of speakers in the mixture (output dimension problem). We propose a novel deep learning framework for speech separation that addresses both of these issues. We use a neural network to project the time-frequency representation of the mixture signal into a high-dimensional embedding space. A reference point (attractor) is created in the embedding space to represent each speaker which is defined as the centroid of the speaker in the embedding space. The time-frequency embeddings of each speaker are then forced to cluster around the corresponding attractor point which is used to determine the time-frequency assignment of the speaker. We propose three methods for finding the attractors for each source in the embedding space and compare their advantages and limitations. The objective function for the network is standard signal reconstruction error which enables end-to-end operation during both training and test phases. We evaluated our system using the Wall Street Journal dataset (WSJ0) on two and three speaker mixtures and report comparable or better performance than other state-of-the-art deep learning methods for speech separation.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings (2020, methodology): “Specifically, we compared our model to i) [22] which trained a separate model to separate a different number of speakers, denoted as Ensemble; ii) [7, 21] which trains one model to separate the maximum number of speakers, denoted as MaxOut.”
Voice Separation with an Unknown Number of Multiple Speakers (2020, methodology): “We compare with the following baseline methods: ADANet (Luo et al., 2018), DPCL++ (Isik et al., 2016), CBLDNN-GAT (Li et al., 2018), TasNet (Luo & Mesgarani, 2018), the Ideal Ratio Mask (IRM), the Ideal Binary Mask (IBM), ConvTasNet (Luo & Mesgarani, 2019), FurcaNeXt (Zhang et al., 2020), and DPRNN ”
Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation (2018, methodology): “ TABLE IV COMPARISON WITH OTHER METHODS ON WSJ0-2MIX DATASET. Method Model size Causal SI-SNRi (dB) SDRi (dB) DPCL++ [5] 13.6M × 10.8 – uPIT-BLSTM-ST [7] 92.7M × – 10.0 DANet [8] 9.1M × 10.5 – ADANet [9] 9.1M × 10.4 10.8 cuPIT-Grid-RD [50] 47.2M × – 10.2 CBLDNN-GAT [12] 39.5M × – 11.0 Chimera++ [10]”
SDR – Half-baked or Well Done? (2018, methodology): “Both SI-SDR and BSS eval ’s SDR have recently been used by various studies [6–9,11,21–23,25,26] in the context of single-channel speaker-independent speech separation on the wsj0-2mix dataset [6], some of these studies reporting both ﬁgures [21–23 ,25].”
End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction (2018, methodology): “We report the performance using scale-invariant SDR (SISDR) [1, 2, 5, 39], as well as the SDR metric computed using the bss eval sources software [40] because it is used by other groups.”
Encoder-Decoder Based Attractor Calculation for End-to-End Neural Diarization (2021, methodology): “While the number of speakers [43] or maximum number of speakers [40] is ﬁxed for the original DANet, in this paper, we calculated a ﬂexible number of attractors without deﬁning them.”
Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective (2018, methodology): “We use scale-invariant signal-to-distortion ratio improvement (SI-SDRi) [33] as the major evaluation metric.”
Recursive speech separation for unknown number of speakers (2019, methodology): “Although the method is shown to work when M = 3 [14, 18], it fails when M < N .”
End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors (2020, background): “Anchored DANet [23] successfully solved the aforementioned problems, but it always requires calculating dot products between all the possible selections of anchors and extracted embeddings even in the inference phase.”
Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation (2020, unknown): “arison with other methods on WSJ0-2mix in SISNR (dB), SDR (dB) and Model Size Method SI-SNR SDR Model Size DPCL++ [27] 10.8 - 13.6M uPIT-BLSTM-ST [12] - 10.0 92.7M Deep Attractor [10] 10.5 - - ADANet [28] 10.4 10.8 9.1M Grid LSTM PIT [29] - 10.2 - ConvLSTM-GAT [30] - 11.0 - Chimera++ [31] 11.5 12.0 ”
Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, background): “In [28], a concept of attractors is introduced to DC to enable ratio masking.”
Ablation Studies in Artificial Neural Networks (2019, background): “…segmentation in the ﬁeld of computer vision (CV) [1–5], speech recognition and speech separation in the ﬁeld of natural language processing (NLP) [6–10] or self-learning agents based on deep reinforcement learning (DRL) for video games [11–14], classic board games [15–17] as well as locomotion and…”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “Two solutions have been proposed in AO settings: permutation invariant training (PIT) [144], [282] and deep clustering (DC) [32], [92], [112], [167].”
FaSNet: Low-Latency Adaptive Beamforming for Multi-Microphone Audio Processing (2019, methodology): “For tasks that take signal quality as an evaluation measure, we use the scale-invariant signal-to-noise ratio (SI-SNR) [34,35] as the training objective: For tasks where frequency-domain output is favored (e.g. ASR), we use mel-spectrogram with scale-invariant mean-square-error (SI-MSE) as the…”
Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, background): “Various followup studies have been reported, including exploration of the different architectures [3, 4, 5] and recipes [6, 7, 8], extension to multi-channel processing [9, 10] and joint modeling with other tasks such as automatic speech recognition (ASR) [11, 12].”