Learning Generator Networks for Dynamic Patterns
By Tian Han, Yang Lu, Jiawen Wu, Xianglei Xing, Ying Nian Wu. 2019
10 citations recorded by the source corpus.
We address the problem of learning dynamic patterns from unlabeled video sequences, either in the form of generating new video sequences, or recovering incomplete video sequences. This problem is challenging because the appearances and motions in the video sequences can be very complex. We propose to use the alternating back-propagation algorithm to learn the generator network with the spatial-temporal convolutional architecture. The proposed method is efficient and flexible. It can not only generate realistic video sequences, but can also recover the incomplete video sequences in the testing stage or even in the learning stage. The proposed algorithm can be further improved by using learned initialization which is useful for the recovery tasks. Further, the proposed algorithm can naturally help to learn the shared representation between different modalities. Our experiments show that our method is competitive with the existing state of the art methods both qualitatively and quantitatively.
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Generative Models
Citations in the corpus
- Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, background): “One generator model [62] assumes a hidden noise vector z Nð0; IdÞ, where Id is the identity matrix of dimension d.”
- Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model (2018, methodology): “Specifically, we start from 10 latent factors of size 1 × 1 × 2 for each video clip and we adopt the same spatial-temporal convolutional transpose generator network as in [53] except we use kernel size 5 for the second layer.”
- Learning Dynamic Generator Model by Alternating Back-Propagation Through Time (2018, methodology): “We can adopt a residual form (He et al. 2016) for F α to model the change of the state vector. x t is the D -dimensional image, which is generated by the d -dimensional hidden state vector s t .”