MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement

By Szu-Wei Fu, Chien-Feng Liao, Yu Tsao, Shou-De Lin. 2019

419 citations recorded by the source corpus.

Adversarial loss in a conditional generative adversarial network (GAN) is not designed to directly optimize evaluation metrics of a target task, and thus, may not always guide the generator in a GAN to generate data with improved metric scores. To overcome this issue, we propose a novel MetricGAN approach with an aim to optimize the generator with respect to one or multiple evaluation metrics. Moreover, based on MetricGAN, the metric scores of the generated data can also be arbitrarily specified by users. We tested the proposed MetricGAN on a speech enhancement task, which is particularly suitable to verify the proposed approach because there are multiple metrics measuring different aspects of speech signals. Moreover, these metrics are generally complex and could not be fully optimized by Lp or conventional adversarial losses.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Real Time Speech Enhancement in the Waveform Domain (2020, background): “Over the last years, there has been a growing interest towards DNN based methods for speech enhancement [34, 35, 36, 37, 20, 38, 39, 22, 40, 7, 41, 8, 42, 21, 37].”
High Fidelity Speech Regeneration with Application to Speech Enhancement (2021, methodology): “Deep generative models such as Generative Adversarial Network (GAN) or WaveNet were also suggested for the task [6, 7, 8].”
Separate Anything You Describe (2023, positive): “CMGAN [77] is a Conformer-based speech enhancement model optimized with MetricGAN [80].”
DeepMMSE: A Deep Learning Approach to MMSE-Based Noise Power Spectral Density Estimation (2020, background): “Recently, perceptually guided training has been exploited to increase speech enhancement performance [59]–[62].”
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement (2021, methodology): “To mitigate this issue, we recently proposed a learning framework where the surrogate loss and enhancement model are alternately updated [15].”
HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement (2022, background): “Audio denoising (Fu et al., 2019; Tagliasacchi et al., 2020) is always a major interest in audio processing community because of its importance and difﬁculty.”
A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, neutral): “A discriminative method minimizes the difference between enhanced and clean speech [13, 16, 17, 42], while generative models are optimized by estimating the distribution of clean signals [51, 80, 86, 106, 108].”
Deep Residual-Dense Lattice Network for Speech Enhancement (2020, methodology): “This was outperformed by another GAN IRM estimator that used multiple objective measures during optimisation (Metric-GAN) (Fu et al. 2019).”
Conditional Diffusion Probabilistic Model for Speech Enhancement (2022, background): “The first category trains discriminative models to minimize the difference between enhanced and clean speech, where the difference can be a point-wise Lp-norm distance [8], or can be computed based on a perceptual metric [9, 10].”