A Factorial Deep Markov Model for Unsupervised Disentangled Representation Learning from Speech

By Sameer Khurana, Shafiq R. Joty, Ahmed M. Ali, James R. Glass. 2019

16 citations recorded by the source corpus.

We present the Factorial Deep Markov Model (FDMM) for representation learning of speech. The FDMM learns disentangled, interpretable and lower dimensional latent representations from speech without supervision. We use a static and dynamic latent variable to exploit the fact that information in a speech signal evolves at different time scales. Latent representations learned by the FDMM outperform a baseline i-vector system on speaker verification and dialect identification while also reducing the error rate of a phone recognition system in a domain mismatch scenario.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units (2021, neutral): “or sequential [28], [30], [32], [33], [43].”
On Generative Spoken Language Modeling from Raw Audio (2021, neutral): “Various constraints can be imposed on the encoded space, such as temporal smoothness (Ebbers et al., 2017; Glarner et al., 2018; Khurana et al., 2019, 2020), discrete-ness (Ondel et al., 2016; van den Oord et al., 2017), and presence of hierarchy (Lee and Glass, 2012; Hsu et al., 2017b).”
ADI17: A Fine-Grained Arabic Dialect Identification Dataset (2020, unknown): “*Work done at MIT CSAIL Broadcast (MGB-3) challenge provided a dataset containing five regional Arabic dialects and resulted in studies covering diverse DID topics such as domain adaptation [2, 3, 4], semi-supervised learning [5, 6, 7, 8], and linguistic feature extraction [1, 9].”
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning (2020, methodology): “They have been used for disentangled representation learning in speech [13, 14, 15].”
The MGB-5 Challenge: Recognition and Dialect Identification of Dialectal Arabic Speech (2019, background): “The previous MGB-3 challenge resulted in studies covering diverse dialect identification topics such as domain adaptation [4, 5], semi-supervised learning [6, 7, 8, 9], and linguistic feature extraction [10, 11].”
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units (2020, methodology): “…2020a), while others have implemented autoen-coder models with various constraints applied to the latent representations, such as discrete-ness (van den Oord et al., 2017; Eloff et al., 2019; Chorowski et al., 2019) or disentanglement (Hsu et al., 2017c; Hsu and Glass, 2018b; Khurana et al., 2019).”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, methodology): “Priors that are autoregressive on the latent variable (ARZ, table 2) take the form p(zt|z1:t−1) and enable stochastic temporal transitions similar to hidden Markov models but with potentially nonlinear transition functions (Chung et al. 2015; Fraccaro et al. 2016; Khurana et al. 2019, 2020).”