MaxEntropy Pursuit Variational Inference

By Evgenii Egorov, Kirill Neklyudov, Ruslan Kostoev, Evgeny Burnaev. 2019

3 citations recorded by the source corpus.

One of the core problems in variational inference is a choice of approximate posterior distribution. It is crucial to trade-off between efficient inference with simple families as mean-field models and accuracy of inference. We propose a variant of a greedy approximation of the posterior distribution with tractable base learners. Using Max-Entropy approach, we obtain a well-defined optimization problem. We demonstrate the ability of the method to capture complex multimodal posterior via continual learning setting for neural networks.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

Citations in the corpus

• BooVAE: Boosting Approach for Continual Learning of VAE (2019, neutral): “Egorov et al. (2019), Locatello et al. (2018) used entropy-based regularization.”