Streaming Variational Inference for Bayesian Nonparametric Mixture Models
By Alex Tank, N. Foti, Emily B. Fox. 2014
39 citations recorded by the source corpus.
In theory, Bayesian nonparametric (BNP) models are well suited to streaming data scenarios due to their ability to adapt model complexity with the observed data. Unfortunately, such benefits have not been fully realized in practice; existing inference algorithms are either not applicable to streaming applications or not extensible to BNP models. For the special case of Dirichlet processes, streaming inference has been considered. However, there is growing interest in more flexible BNP models building on the class of normalized random measures (NRMs). We work within this general framework and present a streaming variational inference algorithm for NRM mixture models. Our algorithm is based on assumed density filtering (ADF), leading straightforwardly to expectation propagation (EP) for large-scale batch inference as well. We demonstrate the efficacy of the algorithm on clustering documents in large, streaming text corpora.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- David M. Blei
- Rajesh Ranganath
- Fei Ye
- Adrian G. Bors
- T. Griffiths
- James McInerney
- Dustin Tran
- Qiang Liu
Citations in the corpus
- Lifelong Generative Modeling (2017, methodology): “Variational methods have been adapted for a streaming setting, e.g: Streaming Variational Bayes Broderick et al. (2013), Streaming Variational Mixture models Tank et al. (2015), and the Population Posterior McInerney et al. (2015).”
- Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, methodology): “As derived in [34], the optimal distribution of the dynamics assignment zn of the nth observation is”
- The Population Posterior and Bayesian Modeling on Streams (2015, methodology): “[22] adapts this framework to nonparametric mixture models.”
- Online gradient-based mixtures for transfer modulation in meta-learning (2018, methodology): “A sequential approximation for nonparametric mixtures with a similar threshold was proposed in Lin (2013) and Tank et al. (2015), where variational Bayes was used instead of point estimation in a DPMM. Unlike traditional nonparametric algorithms, our model does not refine the cluster assignments of…”