Under-Determined Reverberant Audio Source Separation Using a Full-Rank Spatial Covariance Model

By Ngoc Q. K. Duong, Emmanuel Vincent, Rémi Gribonval. IEEE Transactions on Audio, Speech, and Language Processing, 2009

464 citations recorded by the source corpus.

This paper addresses the modeling of reverberant recording environments in the context of under-determined convolutive blind source separation. We model the contribution of each source to all mixture channels in the time-frequency domain as a zero-mean Gaussian random variable whose covariance encodes the spatial characteristics of the source. We then consider four specific covariance models, including a full-rank unconstrained model. We derive a family of iterative expectation-maximization (EM) algorithms to estimate the parameters of each model and propose suitable procedures adapted from the state-of-the-art to initialize the parameters and to align the order of the estimated sources across all frequency bins. Experimental results over reverberant synthetic mixtures and live recordings of speech data show the effectiveness of the proposed approach.

Subfields: Computational Mechanics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/736a64d2953f78e5145235ce32ff0e6aa240509b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Signal Processing

Citations in the corpus

New Era for Robust Speech Recognition (2017, methodology): “In particular, it uses the baseline recognizer from the AMI Meeting Corpus [9, 44], and tests it on the CHiME-3 corpus.”
Multichannel music separation with deep neural networks (2016, unknown): “…each other and follow a multivariate complex-valued zero-mean Gaussian distribution [14], [15], [17], [21]: where v jfn ∈ R + denotes the power spectral density (PSD) of source j for frequency bin f and frame n , and R jf ∈ C I × I is the spatial covariance matrix of source j for frequency bin f .”
Direction of Arrival Based Spatial Covariance Model for Blind Sound Source Separation (2014, unknown): “A direct estimation of the source SCM and variances at each frequency is done for example in [34], but it again requires solving the frequency-wise permutation.”
Multichannel Audio Source Separation With Deep Neural Networks (2016, methodology): “We study the impact of different design choices on the performance, including the cost function used for the training of DNNs and the number of EM iterations.”
An analysis of environment, microphone and data simulation mismatches in robust speech recognition (2017, unknown): “…us-ing the DNN-based multichannel source separation technique of Nugraha et al. (2016a), which is a variant of the one of Sivasankaran et al. (2015) that relies on exact EM updates for the spatial covariance matrices (Duong et al., 2010) instead of the weighted EM updates of Liutkus et al. (2015).”
2.5D Visual Sound (2018, background): “Separation becomes easier when multiple channels are observed using multiple microphones [29, 49, 9] or binaural audio [48, 8, 50].”
Learning to Separate Object Sounds by Watching Unlabeled Video (2018, methodology): “Some methods assume access to multiple microphones, which facilitates separation [19, 51, 75].”
Mmdenselstm: An Efficient Combination of Convolutional and Recurrent Neural Networks for Audio Source Separation (2018, methodology): “Various approaches have been introduced such as lo-cal Gaussian modeling [1, 2], non-negative factorization [3– 5], kernel additive modeling [6] and their combinations [7–9].”
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis (2022, background): “While early formulations of the problem assumed input from multiple microphones [9, 66], recent approaches have also considered the monaural setting [23,50,53,54].”
A Survey of Sound Source Localization with Deep Learning Methods (2021, background): “, Dvorkind and Gannot [2005], Li et al. [2016c], Laufer-Goldshtein et al. [2020], Li et al. [2016b], they are all intrinsically based on certain (more or less plausible) modeling assumptions, which can limit their effectiveness when exposed to the complexity of real-world acoustics.”
Multi-Scale multi-band densenets for audio source separation (2017, methodology): “Various approaches have been introduced so far such as local Gaussian modeling [1, 2], non-negative factorization [3–5], kernel additive modeling [6] and combinations of those approaches [7–9].”
A General Flexible Framework for the Handling of Prior Information in Audio Source Separation (2012, methodology): “It can be easily shown that the model structures considered in [2]–[19] are particular instances of the proposed genera l formulation.”
Speech Processing for Digital Home Assistants: Combining signal processing with deep-learning techniques (2019, methodology): “is the a priori probability that an observation belongs to mixture component k and ( ) p y , t f k ;i is an appropriate component distribution with parameters k i [17]–[19].”
The 2018 Signal Separation Evaluation Campaign (2018, background): “…given by: where C j ( f, t ) is the I × I covariance matrix for source j at TF bin ( f, t ) and C x = j C j . local In the classical Gaussian model [6], the further parameterization C j ( f, t ) = v j ( f, t ) R j ( f ) is picked, with R j being the I × I spatial covariance matrix , encoding the…”
A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation (2017, unknown): “Many approaches based on ML and alternative statistical estimation criteria such as maximum a posteriori (MAP) and variational Bayes (VB) have then been proposed [53], [110]–[113], [115], [116], [159], [227]–[237].”
Determined Blind Source Separation Unifying Independent Vector Analysis and Nonnegative Matrix Factorization (2016, methodology): “There have been several multichannel extensions of NMF [27]–[30] and related methods [31], [32].”