Stochastic Conditional Generative Networks with Basis Decomposition

By Ze Wang, Xiuyuan Cheng, Guillermo Sapiro, Qiang Qiu. 2019

19 citations recorded by the source corpus.

While generative adversarial networks (GANs) have revolutionized machine learning, a number of open questions remain to fully understand them and exploit their power. One of these questions is how to efficiently achieve proper diversity and sampling of the multi-mode data space. To address this, we introduce BasisGAN, a stochastic conditional multi-mode image generator. By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of basis elements, we learn a plug-and-played basis generator to stochastically generate basis elements, with just a few hundred of parameters, to fully embed stochasticity into convolutional filters. By sampling basis elements instead of filters, we dramatically reduce the cost of modeling the parameter space with no sacrifice on either image diversity or fidelity. To illustrate this proposed plug-and-play framework, we construct variants of BasisGAN based on state-of-the-art conditional image generation networks, and train the networks by simply plugging in a basis generator, without additional auxiliary components, hyperparameters, or training objectives. The experimental success is complemented with theoretical results indicating how the perturbations introduced by the proposed sampling of basis elements can propagate to the appearance of generated images.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Mengyao Zhai
2. Lei Chen
3. Greg Mori
4. Qiang Qiu
5. Yan Xu
6. Eric I-Chao Chang
7. Shengyu Zhao
8. Ze Wang

In Statistical and Nonlinear Physics

1. Greg Mori
2. Qiang Qiu
3. Ze Wang
4. Wei Chen
5. Yan Xu
6. Zichen Miao

In Generative Models

1. Greg Mori
2. Mengyao Zhai
3. Lei Chen
4. Eric I-Chao Chang
5. Yilun Sheng
6. Qiang Qiu
7. Ze Wang
8. Yue Dong

Citations in the corpus

• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, methodology): “We follow the common setting (DeVries et al., 2019; Wang et al., 2019b) on the edges to photos datasets (Isola et al., 2017) at 256×256 resolution, where FID samples once per validation image (200 in total) and the training set is used as the reference distribution; LPIPS measures the…”
• Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation (2021, methodology): “For instance, for U-Net generator [17], only the decoder would be dynamic, parameterized as hypernetworks while the encoder remains deterministic [24]; for Resnet generator [19, 42], there would be a fixed sub-model while only last few layers are parameterized as hypernetworks [34].”
• Continual Learning with Filter Atom Swapping (2022, methodology): “This work further inspires other works in domain adaptation (Wang et al., 2020b), adaptive convolution (Wang et al., 2021b; 2020a), image generation (Wang et al., 2021a; 2019), video understanding (Miao et al., 2021), rotation equivariance (Cheng et al., 2018) and graph convolution (Cheng et al.,…”