Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet

By Jianwen Xie, Song-Chun Zhu, Ying Nian Wu. 2016

104 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, neutral): “By the known connection between score matching and energy-based modeling, our work could have implications for other recent work on energy-based models [67–69, 12, 70, 13, 11, 41, 17, 8].”
Residual Energy-Based Models for Text Generation (2020, methodology): “…gradient-based MCMC approaches like Hybrid Monte Carlo (Teh et al., 2003) and Langevyn dynamics (Ranzato et al., 2007; Du & Mordatch, 2019; Xie et al., 2016; 2017; 2019; 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text applications.”
Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, background): “This paper is an expanded version of our conference paper in [39].”
On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, background): “[40, 39] explore an adversarial interpretation of ML learning.”
MoCoGAN: Decomposing Motion and Content for Video Generation (2017, methodology): “Due to limitations in computation, data, and modeling tools, early video generation works focused on generating dynamic texture patterns [35, 42, 9, 43].”
Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks (2016, background): “More recently, works in image synthesis have begun to produce impressive results by training variants of neural network structures to produce novel images [Gregor et al., 2015, Xie et al., 2016].”
Cooperative Training of Descriptor and Generator Networks (2016, methodology): “We can learn to generate video sequences by cooperative training of a spatial-temporal descriptor network [41] and a spatial-temporal generator network.”
Learning Generator Networks for Dynamic Patterns (2019, background): “Xie [34] trained the spatial-temporal energy model and also showed sharp generation results on dynamic textures.”
Learning Latent Space Energy-Based Prior Model (2020, background): “The maximum likelihood learning of EBM can be interpreted as an adversarial scheme [58, 22, 33, 60], where the MCMC serves as the generator and the energy function serves as an evaluator.”
Improved Contrastive Divergence Training of Energy Based Models (2020, background): “…generation (Hinton, 1999; Du et al., 2020a), memory modeling (Bartunov et al., 2019), text generation (Deng et al., 2020), video generation (Xie et al., 2017), reinforcement learning (Haarnoja et al., 2017; Du et al., 2019), continual learning (Li et al., 2020), protein design and folding…”
Residual Energy-Based Models for Text (2021, methodology): “…gradient-based MCMC approaches like Hybrid Monte Carlo (Teh et al., 2003) and Langevyn dynamics (Ranzato et al., 2007; Du and Mordatch, 2019; Xie et al., 2016, 2017, 2019, 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text applications.”
Stochastic Image-to-Video Synthesis using cINNs (2021, methodology): “[85] did not converge when trained on all test samples.”
Flow-Grounded Spatial-Temporal Video Prediction from Still Images (2018, methodology): “Both traditional methods based on linear dynamical systems [26,27] and neural network based methods [28] require learning a model for each sequence example.”
Energy-Based Models for Text (2020, methodology): “…gradient-based MCMC approaches like Hybrid Monte Carlo (Teh et al., 2003) and Langevyn dynamics (Ranzato et al., 2007; Du and Mordatch, 2019; Xie et al., 2016, 2017, 2019, 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text applications.”
Two-Stream Convolutional Networks for Dynamic Texture Synthesis (2017, methodology): “[48] proposed a spatiotemporal generative model where each dynamic texture is modelled as a random field defined by multiscale, spatiotemporal ConvNet filter responses and dynamic textures are realized by sampling the model.”