Rethinking LDA: Moment Matching for Discrete ICA

By Anastasia Podosinnikova, Francis R. Bach, Simon Lacoste-Julien. 2015

27 citations recorded by the source corpus.

We consider moment matching techniques for estimation in latent Dirichlet allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techniques such as joint diagonalization of tensors to improve over existing methods based on the tensor power method. In an extensive set of experiments on both synthetic and real datasets, we show that our new combination of tensors and orthogonal joint diagonalization techniques outperforms existing moment matching methods.

Subfields: Artificial Intelligence, Computational Mathematics, Probabilistic & Bayesian Methods, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Simon Lacoste-Julien
2. Francis R. Bach
3. Peter Richtárik
4. Anastasia Podosinnikova
5. Martin Jaggi
6. Nicolas Loizou
7. Zeyuan Allen-Zhu
8. Alex Smola

In Computational Mathematics

1. Simon Lacoste-Julien
2. Anastasia Podosinnikova
3. Francis R. Bach

In Probabilistic & Bayesian Methods

1. Simon Lacoste-Julien
2. Anastasia Podosinnikova
3. Francis R. Bach
4. Michael W. Mahoney
5. Ivan Laptev
6. Armand Joulin
7. Piotr Bojanowski
8. A. Alaoui

In Signal Processing

1. Simon Lacoste-Julien
2. Francis R. Bach
3. Anastasia Podosinnikova
4. Arthur Gretton
5. Bernhard Schölkopf
6. Sebastian U. Stich
7. Kenji Fukumizu
8. Julien Mairal

Citations in the corpus

• Beyond CCA: Moment Matching for Multi-View Models (2016, methodology): “…(e.g., Arora & Kannan, 2005; Hsu & Kakade, 2013), latent Dirichlet allocation and topic models (Arora et al., 2012; 2013; Anandkumar et al., 2012; Podosinnikova et al., 2015), supervised topic models (Wang & Zhu, 2014), Indian buffet process inference (Tung & Smola, 2014), stochastic languages…”