Alternating Back-Propagation for Generator Network

By Tian Han, Yang Lu, Song-Chun Zhu, Ying Nian Wu. 2016

139 citations recorded by the source corpus.

This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating back-propagation algorithm iterates the following two steps: (1) Inferential back-propagation, which infers the latent factors by Langevin dynamics or gradient descent. (2) Learning back-propagation, which updates the parameters given the inferred latent factors by gradient descent. The gradient computations in both steps are powered by back-propagation, and they share most of their code in common. We show that the alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds. Moreover, it can also be used to learn from incomplete or indirect training data.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, methodology): “To address the inefficiency of MCMC sampling, [36] developed a multi-grid sampling and learning method for the energy-based generative ConvNet, and [37] proposed to recruit a top-down generator [38] serving as an approximate sampler of the energy-based generative ConvNet model.”
Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model (2019, methodology): “For GAN, we perform Langevin inference of latent variables [16, 46].”
Unrolled Generative Adversarial Networks (2016, background): “More recently, (Belanger & McCallum, 2015; Han et al., 2016; Andrychowicz et al., 2016) backpropagate through optimization procedures in contexts unrelated to GANs or minimax games.”
Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model (2018, methodology): “We have learned the generator using such learning method [13].”
DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data (2017, background): “[9] propose to alternate between training the latent factors and the generator parameters.”
Cooperative Training of Descriptor and Generator Networks (2016, methodology): “ the observed examples, but it requires inferring the latent factors for each observed example. 1.4 Cooperative training via MCMC teaching The two algorithms can operate separately on their own [15], [25]. But just like the case with the student and the advisor, they beneﬁt from cooperating with eac”
Cooperative Learning of Energy-Based Model and Latent Variable Model via MCMC Teaching (2018, methodology): “Algorithm G (Han et al. 2017) iterates the following two steps after initializing α and {Xi, i = 1, .”
Learning Dynamic Generator Model by Alternating Back-Propagation Through Time (2018, methodology): “(2) The learning of the dynamic model is end-to-end, which is different from (Han et al. 2017)’s treatment.”
Learning Latent Space Energy-Based Prior Model (2020, methodology): “More recently in [20, 44], the generator model has also been learned by maximum likelihood without resorting to a complementary model, where the inference is carried out by Markov chain Monte Carlo (MCMC) such as the Langevin dynamics.”
Learning Generator Networks for Dynamic Patterns (2019, methodology): “A major class of models on dynamic textures focus on the state space auto-regressive model for synthesis [5, 9, 38, 4] and classification [2, 36].”
Building a Telescope to Look Into High-Dimensional Image Spaces (2018, background): “One way to restrict proposals to a well-formed region of the image landscape is to use a generator ConvNet [14, 20] as the proposal mechanism.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “The inference network can be removed by assigning latent vectors to data points, then optimising them with Langevin dynamics or gradient descent, during training; although this allows fast training, convergence for unseen samples is not guaranteed and there is still a large discrepancy between the t”
Learning Deep Latent Gaussian Models with Markov Chain Monte Carlo (2017, methodology): “More recently, Han et al. (2017) proposed using Metropolis-adjusted Langevin (i.e., HMC with one leapfrog step) to fit DLGMs in a stochastic EM framework.”
Controllable and Compositional Generation with Latent-Space Energy-Based Models (2021, background): “Also, many other works have built EBMs or score-based models in the latent space of generative models, most of which, however, focus on improving generation quality with EBMs as a structural prior [2, 4, 19, 56, 55, 58, 53].”