Deep learning for monaural speech separation

By Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2014

458 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/ca7f25ac119e2d706e51a63f6178f1547a863bcc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement (2020, background): “Hence, early studies only focus on magnitude related training target while ignoring phase [15–17], resynthesizing the estimated speech by simply applying estimated magnitude with the noisy speech phase.”
Self-Supervised Generation of Spatial Audio for 360 Video (2018, background): “For example, [18] proposes a recurrent neural-network for monaural separation of two speakers, [1, 12, 11] seek to isolate sound sources by leveraging synchronized visual information in addition to the audio input, and [42] studies a wide range of frequency-based separation methods.”
Learning to Separate Object Sounds by Watching Unlabeled Video (2018, methodology): “Popular approaches include Independent Component Analysis (ICA) [36], sparse decomposition [78], Computational Auditory Scene Analysis (CASA) [21], non-negative matrix factorization (NMF) [23,24,46,69], probabilistic latent variable models [34,61], and deep learning [32,35].”
A Survey on Deep Learning (2018, background): “Huang et al. [66] study the monaural source separation in deep learning.”
Deep neural network based instrument extraction from music (2015, result): “This is in contrast to other proposed DNN approaches for BSS of music which are supervised in the sense that they assume to have more knowledge about the signals [10–12].”
Discriminatively trained recurrent neural networks for single-channel speech separation (2014, methodology): “As a result, RNNs are often not able to outperform DNNs in practical speech processing tasks [9], [17].”
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency (2021, background): “While early work assumes access to multiple microphones to facilitate separation [53, 78, 18], some methods tackle the “blind” separation problem with monaural audio [65, 68, 69], including recent deep learning approaches [37, 73, 33, 40, 48].”
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis (2022, methodology): “This includes monau-ral speech separation methods, which address the problem of separating a mixture of speakers from a single audio track [23, 36, 60, 67], as well as monaural speech enhancement methods, which tackle the problem of removing non-speech background sounds [8, 30, 42, 55, 56, 63] and…”
Polyphonic sound event detection using multi label deep neural networks (2015, background): “In the end, our conclusions on the topic are given in Section 6.”
Universal Sound Separation (2019, background): “For separation of multiple speakers, the state of the art has progressed from speaker-dependent separation [13], where models are trained on individual speakers or speaker combinations, to speaker-independent speech separation [6–8], where the system has to be flexible enough to separate unknown spe”
Monoaural Audio Source Separation Using Deep Convolutional Neural Networks (2017, methodology): “After some initial experimentation, we observed that an additional loss term, Ldiff , representing the difference between the estimated sources, as used by [8], improved the performance of the system.”
Improving Robustness of Deep Neural Network Acoustic Models via Speech Separation and Joint Adaptive Training (2015, methodology): “For the DNN-JAT system that we present here, it is assumed that there is not a lot of channel mismatch between training and testing, and that speech separation primarily addresses background noise.”
Improved MVDR Beamforming Using Single-Channel Mask Prediction Networks (2016, background): “Recent work on speech enhancement and source separation also indicate that deep neural networks [7, 8, 9, 10, 11], especially deep recurrent neural networks such as long short-term memory (LSTM) networks yield much better performance than the closest competitor in speech enhancement of single-channe”
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures (2014, methodology): “Here, based on our experience with model-based approaches, we train the masking function such that, when applied to the mixture, it best reconstructs the clean speech, which was also proposed in [29].”
End-To-End Source Separation With Adaptive Front-Ends (2017, methodology): “Several neural network (NN) architectures and methods have been proposed for supervised single-channel source separation [1, 2, 3, 4].”
Single-Channel Multi-talker Speech Recognition with Permutation Invariant Training (2017, result): “In the simplest architecture, which is denoted as Arch#1 and illustrated in Figure 1(a), feature separation can be considered as a multi-class regression problem, similar to many previous works [29], [30], [31], [32], [33], [34].”
Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, methodology): “Many deep learning based speaker separation systems [7], [17], [44] address this problem in the T-F domain, where STFT is calculated using an analysis window w(n) with FFT length N and frame shift R:”