Exponential Family Estimation via Adversarial Dynamics Embedding

By Bo Dai, Z. Liu, Hanjun Dai, Niao He, Arthur Gretton, Le Song, Dale Schuurmans. 2019

58 citations recorded by the source corpus.

We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks. We exploit the primal-dual view of the MLE with a kinetics augmented model to obtain an estimate associated with an adversarial dual sampler. To represent this sampler, we introduce a novel neural architecture, dynamics embedding, that generalizes Hamiltonian Monte-Carlo (HMC). The proposed approach inherits the flexibility of HMC while enabling tractable entropy estimation for the augmented model. By learning both a dual sampler and the primal model simultaneously, and sharing parameters between them, we obviate the requirement to design a separate sampling procedure once the model has been trained, leading to more effective learning. We show that many existing estimators, such as contrastive divergence, pseudo/composite-likelihood, score matching, minimum Stein discrepancy estimator, non-local contrastive objectives, noise-contrastive estimation, and minimum probability flow, are special cases of the proposed approach, each expressed by a different (fixed) dual sampler. An empirical investigation shows that adapting the sampler during MLE can significantly improve on state-of-the-art estimators.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Reinforcement Learning via Fenchel-Rockafellar Duality (2020, background): “…machine learning applications, including variational inference (Wainwright and Jordan, 2003), generative model estimation (Nowozin et al., 2016; Dai et al., 2019), imitation learning (Ke et al., 2019; Ghasemipour et al., 2019; Kostrikov et al., 2019), and reinforcement learning (Nachum et al.,…”
KALE: When Energy-Based Learning Meets Adversarial Training (2020, methodology): “(Dai et al., 2019a;b) consider the task of MLE using an inﬁnite-dimensional set of energies.”
Training Deep Energy-Based Models with f-Divergence Minimization (2020, methodology): “…function (an integral over the sample space), so far within the family of f -divergences, only KL divergence proved to be tractable to optimize with methods such as contrastive divergence (Hinton, 2002), doubly dual embedding (Dai et al., 2018) and adversarial dynamics embedding (Dai et al., 2019).”
Overcoming Catastrophic Forgetting by Bayesian Generative Regularization (2019, methodology): “In fact, new objective is also related to contrastive loss (Chen et al., 2020; Dai et al., 2019), where we use the synthesis samples as negative samples.”
Overcoming Catastrophic Forgetting by Generative Regularization (2019, unknown): “Our work follows the derivation in (Dai et al., 2019).”
Generalized Energy Based Models (2020, methodology): “Thus, various methods have been proposed to address this (Hinton, 2002; Hyvärinen, 2005; Gutmann and Hyvärinen, 2012; Dai et al., 2019a;b).”
Diffusion policy: Visuomotor policy learning via action diffusion (2023, unknown): “Implicit policies (Florence et al., 2021; Jarrett et al., 2020) de ﬁ ne distributions over actions by using Energy-Based Models (EBMs) (Dai et al., 2019; Du et al., 2020; Du and Mordatch, 2019; Grathwohl et al., 2020; LeCun et al., 2006).”
Learning Universal Policies via Text-Guided Video Generation (2023, methodology): “We emphasize that the UPDP formulation is also compatible with other probabilistic models, such as a variational autoencoder (Kingma & Welling, 2013), energy-based model (Du & Mordatch, 2019; Dai et al., 2019), or generative adversarial network (Goodfel-low et al., 2014).”
Improved Contrastive Divergence Training of Energy Based Models (2020, background): “…issue, a separate line of work utilizes an auxiliary network to amortize the negative portions of the sampling procedure (Kim & Bengio, 2016; Kumar et al., 2019; Han et al., 2019; Xie et al., 2018a; Song & Ou, 2018; Dai et al., 2019; Che et al., 2020; Grathwohl et al., 2020a; Arbel et al., 2020).”
Model Based Planning with Energy Based Models (2019, background): “Energy-based models have seen success in other applications, such as natural image modeling [10, 18].”
How to Train Your Energy-Based Models (2021, methodology): “See Dai et al. (2019b) for an example on designing q φ ( x ) and training EBMs with Eq.”