A-NICE-MC: Adversarial Training for MCMC

By Jiaming Song, Shengjia Zhao, Stefano Ermon. 2017

114 citations recorded by the source corpus.

Existing Markov Chain Monte Carlo (MCMC) methods are either based on general-purpose and domain-agnostic schemes which can lead to slow convergence, or hand-crafting of problem-specific proposals by an expert. We propose A-NICE-MC, a novel method to train flexible parametric Markov chain kernels to produce samples with desired properties. First, we propose an efficient likelihood-free adversarial training method to train a Markov chain and mimic a given data distribution. Then, we leverage flexible volume preserving flows to obtain parametric kernels for MCMC. Using a bootstrap approach, we show how to train efficient Markov chains to sample from a prescribed posterior distribution by iteratively improving the quality of both the model and the samples. A-NICE-MC provides the first framework to automatically design efficient domain-specific MCMC proposals. Empirical results demonstrate that A-NICE-MC combines the strong guarantees of MCMC with the expressiveness of deep neural networks, and is able to significantly outperform competing methods such as Hamiltonian Monte Carlo.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, methodology): “Other methods for learning transition operators of Markov chains include infusion training [2], variational walkback [13], generative stochastic networks [1], and others [47, 51, 33, 39].”
Stochastic Normalizing Flows (2020, background): “It has been demonstrated that combining learnable proposals/transformations with stochastic sampling techniques can improve expressiveness of the proposals (Salimans et al., 2015; Levy et al., 2017; Song et al., 2017; Hoffman et al., 2019; Hoffman, 2017).”
Generative Modeling by Estimating Gradients of the Data Distribution (2019, methodology): “Our approach has some similarities with methods that learn the transition operator of a Markov chain for sample generation [4, 51, 5, 16, 52].”
Generalizing Hamiltonian Monte Carlo with Neural Networks (2017, methodology): “These additional statistics could also include learned statistics of the samples, combining benefits of the adversarial approach of (Song et al., 2017) with the current work.”
Denoising Diffusion Implicit Models (2020, methodology): “Our work is based on a large family of existing methods on learning generative models as transition operators of Markov chains (Sohl-Dickstein et al., 2015; Bengio et al., 2014; Salimans et al., 2014; Song et al., 2017; Goyal et al., 2017; Levy et al., 2017).”
Batch Stationary Distribution Estimation (2020, background): “thods [Oates et al., 2017] require additional information about the target distribution [Robert and Casella, 2004]. Recent advances have also shown that learning parametric samplers can be benecial [Song et al., 2017, Li et al., 2019], but require the potential function. In contrast, VPM directly le”
GenDICE: Generalized Offline Estimation of Stationary Values (2020, background): “Amortized SVGD (Wang & Liu, 2016) and Adversarial MCMC (Song et al., 2017; Li et al., 2019) alleviate this issue via combining with neural network, but they still interact with the transition operator directly, i.e., in an on-policy setting.”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “One example is A-NICE-MC (Song et al., 2017), which is similar to HMC”
Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models (2017, background): “Directed latent variable models allow for efficient ancestral sampling and hence these models can also be trained using other divergences, e.g., adversarially (Mescheder, Nowozin, and Geiger 2017a; Mao et al. 2017; Song, Zhao, and Ermon 2017).”
Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models (2020, background): “Salimans et al. (2015) on the other hand propose a hierarchical model inspired by the Hamiltonian dynamic, and Song et al. (2017); Levy et al. (2018) generalize Hamiltonian Monte Carlo (HMC) with trainable neural components.”
Stein Neural Sampler (2018, unknown): “Song et al. (2017) proposed A-NICE-MC, where the proposal distribution in MCMC is, instead of domain-agnostic, adversarially trained using neural networks.”
Exponential Family Estimation via Adversarial Dynamics Embedding (2019, methodology): “The proposed algorithm shares some similarities with recent advances in meta learning for sampling (Levy et al., 2018; Feng et al., 2017; Song et al., 2017; Gong et al., 2019), in which the sampler is parametrized via neural network and will be learned through certain objectives.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “samplers with neural networks can also improve the efficiency of sampling [62], [88], [134], [199], [215].”
Neural Network Renormalization Group (2018, background): “[79] Jiaming Song, Shengjia Zhao, and Stefano Ermon, “A-NICEMC: Adversarial Training for MCMC,” arXiv:1706.”
MintNet: Building Invertible Neural Networks with Masked Convolutions (2019, methodology): “They have been employed to investigate representations of deep classiﬁers [15], understand the cause of adversarial examples [14], learn transition operators for MCMC [28, 18], create generative models that are directly trainable by maximum likelihood [6, 5, 24, 16, 9, 1], and perform approximate in”
Kernel Implicit Variational Inference (2017, background): “Existing implicit posteriors (Mescheder et al., 2017; Song et al., 2017) based on traditional fully-connected neural networks cannot handle such a high-dimensional output space.”
Metropolis-Hastings Generative Adversarial Networks (2018, methodology): “Song et al. [19] use a GAN-like procedure to train a RealNVP [6] MCMC proposal for sampling an externally provided target p (cid:63) .”