Nonparametric Estimation of Multi-View Latent Variable Models

By Le Song, Anima Anandkumar, Bo Dai, Bo Xie. 2013

46 citations recorded by the source corpus.

Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel method for learning multi-view latent variable models, allowing each mixture component to be nonparametric. The key idea of the method is to embed the joint distribution of a multi-view latent variable into a reproducing kernel Hilbert space, and then the latent parameters are recovered using a robust tensor power method. We establish that the sample complexity for the proposed method is quadratic in the number of latent components and is a low order polynomial in the other relevant parameters. Thus, our non-parametric tensor approach to learning latent variable models enjoys good sample and computational efficiencies. Moreover, the non-parametric tensor power method compares favorably to EM algorithm and other existing spectral algorithms in our experiments.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Beyond CCA: Moment Matching for Multi-View Models (2016, neutral): “Importantly, the joint diagonalization problem we deal with in this paper is conceptually different from the one considered, e.g., by Kuleshov et al. (2015) (and references therein) and, therefore, the respective algorithms are not applicable here.”
The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, methodology): “[39] proves identifiability for multi-view, latent variable models, unifying previously proposed spectral techniques [2].”
Scale Up Nonlinear Component Analysis with Doubly Stochastic Gradients (2015, methodology): “Nonparametric Latent Variable Model We learn latent variable models with DSGD-KSVD using one million data points [22], achieving higher quality solutions compared with two other approaches.”
Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods (2017, unknown): “For the first term in (48), from Lemmata 4 and 5 of Song et al. (2013), we have ‖T̃ (W − Ŵ ,W − Ŵ ,W − Ŵ )‖ ≤ 64‖E2‖ 3 λ̃3.”
Reinforcement Learning of POMDPs using Spectral Methods (2016, methodology): “Notice that the first two terms are basically the same as in thebounds for spectral methods applied to HMM (Song et al., 2013), while the dependency onπmin is specific to the POMDP case.”