Unsupervised Learning of Disentangled Speech Content and Style Representation

By Andros Tjandra, Ruoming Pang, Yu Zhang, Shigeki Karita. 2020

21 citations recorded by the source corpus.

We present an approach for unsupervised learning of speech representation disentangling contents and styles. Our model consists of: (1) a local encoder that captures per-frame information; (2) a global encoder that captures per-utterance information; and (3) a conditional decoder that reconstructs speech given local and global latent variables. Our experiments show that (1) the local latent variables encode speech contents, as reconstructed speech can be recognized by ASR with low word error rates (WER), even with a different global encoding; (2) the global latent variables encode speaker style, as reconstructed speech shares speaker identity with the source utterance of the global encoding. Additionally, we demonstrate an useful application from our pre-trained model, where we can train a speaker recognition model from the global latent variables and achieve high accuracy by fine-tuning with as few data as one label per speaker.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Text-Free Prosody-Aware Generative Spoken Language Modeling (2021, background): “…is increasing interest in exploring speech pre-training using large quantities of unlabeled speech data (Chung et al., 2019; Schneider et al., 2019; Kharitonov et al., 2021; Baevski et al., 2020; Hsu et al., 2021c; Liu et al., 2020; Ling and Liu, 2020; Tjandra et al., 2020; Hsu et al., 2021b,a).”