Online Variational Inference for the Hierarchical Dirichlet Process

By Chong Wang, John William Paisley, David M. Blei. International Conference on Artificial Intelligence and Statistics, 2011

407 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/93792105974f4d42c83172c4fc9f24be77fe781b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Scalable inference of topic evolution via models for latent geometric structures (2018, methodology): “Our approach is able to perform dynamic nonparametric topic inference on 3 million documents in 20 minutes, which is significantly faster than prior static online and/or distributed topic modeling algorithms [16, 12, 25, 6, 5].”
Streaming dynamic and distributed inference of latent geometric structures (2018, methodology): “…to perform dynamic nonparametric topic inference on 3 million documents in 12 minutes, which is significantly faster than prior static online and/or distributed topic modeling algorithms (Newman et al., 2008; Hoffman et al., 2010; Wang et al., 2011; Bryant & Sudderth, 2012; Broderick et al., 2013).”
Discovering Discrete Latent Topics with Neural Variational Inference (2017, background): “Amongst our selection of unbounded topic models, we compare our truncation-free RSB model, which applies an RNN to dynamically increase the active topics ( γ is empirically set as 5 e − 5 ), with the traditional non-parametric HDP topic model (Teh et al., 2006).”
A General Method for Robust Bayesian Modeling (2015, methodology): “We follow the metric used in recent topic modeling literature (Blei and Lafferty, 2007; Asuncion et al., 2009; Wang et al., 2011; Hoffman et al., 2013), where we hold out part of a document and predict its remainder.”
Automated Variational Inference in Probabilistic Programming (2013, methodology): “…inference algorithms, in particular in the work of Blei et al. in stochastic variational inference (for instance, online LDA [19], on-line HDP [20], and more generally under conjugacy assumptions [21]), as a way to reﬁne estimates of latent variable distributions without processing all the…”
Advances in Variational Inference (2017, background): “Stochastic Variational Inference amounts to applying stochastic optimization to the objective function encountered in VI [65], [67], [71], [209], thereby scaling VI to very large datasets.”
Content-based recommendations with Poisson factorization (2014, methodology): “We use the predictive approach to evaluating model ﬁtness [7], comparing the predictive accuracy of the CTPF coordinate ascent algorithm in Figure 2 to collaborative topic regression (CTR) [21].”
Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation (2013, background): “[23] extended this framework to the non-parametric realm, and developed an online learning algorithm for HDP [18].”
Dynamic Language Models for Streaming Text (2014, background): “Strictly speaking, online learning is distinct from stochastic learning, which for language models built on massive datasets has been explored by Hoffman et al. (2013) and Wang et al. (2011).”
Streaming Variational Bayes (2013, unknown): “An exception to this statement is provided by [3–5], who have shown that a class of approximation methods known as variational Bayes (VB) [6] can be usefully deployed for large-scale data sets.”
Active Multitask Learning Using Both Latent and Supervised Shared Topics (2014, methodology): “It uses a modified stick breaking construction of Hierarchical Dirichlet Process (HDP), recently introduced in [40], to make variational inference feasible.”
Beta-Negative Binomial Process and Poisson Factor Analysis (2011, methodology): “The performance measure is similar to those used in Asuncion et al. (2009); Wallach et al. (2009); Wang et al. (2011).”
Variance Reduction for Stochastic Gradient Optimization (2013, methodology): “Similar ideas can be used in deriving control variates for hierarchical Dirichlet process [12, 13] and nonnegative matrix factorization [14].”
Variational Bayesian Inference with Stochastic Search (2012, methodology): “In addition, recent development of the theory has extended the method to online inference and stochastic optimization settings, making variational Bayes a viable approach for Bayesian learning with massive data sets (Hoﬀman et al., 2010; Wang et al., 2011).”
Full-Text or Abstract? Examining Topic Coherence Scores Using Latent Dirichlet Allocation (2017, unknown): “[27]– [29]).”
Stochastic Variational Inference for Bayesian Time Series Models (2014, methodology): “However, while SVI has been studied extensively for topic models (Hoffman et al., 2010; Wang et al., 2011; Bryant & Sudderth, 2012; Wang & Blei, 2012; Ranganath et al., 2013; Hoffman et al., 2013), it has not been applied to time series.”
A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models (2012, methodology): “Similar approaches to online variational inference have been developed for latent Dirichlet allocation by Hoﬀman et al. (2010) and for the hierarchical Dirichlet process by Wang et al. (2011).”