Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

By Siyuan Feng, Tan Lee, Zhiyuan Peng. 2019

21 citations recorded by the source corpus.

This study addresses the problem of unsupervised subword unit discovery from untranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech 2019, building text-to-speech systems without text labels. In this work, unit discovery is formulated as a pipeline of phonetically discriminative feature learning and unit inference. One major difficulty in robust unsupervised feature learning is dealing with speaker variation. Here the robustness towards speaker variation is achieved by applying adversarial training and FHVAE based disentangled speech representation learning. A comparison of the two approaches as well as their combination is studied in a DNN-bottleneck feature (DNN-BNF) architecture. Experiments are conducted on ZeroSpeech 2019 and 2017. Experimental results on ZeroSpeech 2017 show that both approaches are effective while the latter is more prominent, and that their combination brings further marginal improvement in across-speaker condition. Results on ZeroSpeech 2019 show that in the ABX discriminability task, our approaches significantly outperform the official baseline, and are competitive to or even outperform the official topline. The proposed unit sequence smoothing algorithm improves synthesis quality, at a cost of slight decrease in ABX discriminability.

Subfields: Representation & Self-Supervised Learning, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Speech & Audio

Citations in the corpus

Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, methodology): “To obtain discrete representations, a post-hoc clustering step can be applied to the continuous representations (Kamper et al., 2017b; Feng et al., 2019).”
On Generative Spoken Language Modeling from Raw Audio (2021, neutral): “…representations in an unsupervised manner, has attracted attention from both the machine learning and the speech communities (Liu et al., 2019a; Feng et al., 2019; Nayak et al., 2019; Tjan-dra et al., 2019; Schneider et al., 2019; Baevski et al., 2020a; Chen and Hain, 2020; Morita and Koda,…”
The Zero Resource Speech Challenge 2019: TTS without T (2019, background): “FE [27] no no Ossian LI [28] yes yes Inversion”
Towards Unsupervised Learning of Speech Features in the Wild (2021, methodology): “While such techniques have been applied to unsupervised representation learning [23, 24, 25] to our knowledge, they have not been tested with predictive learning.”
Unsupervised Acoustic Unit Representation Learning for Voice Conversion using WaveNet Auto-encoders (2020, methodology): “[20] used a disentanglement learning model [3] combined with the DNN bottleneck feature learning system.”