Whitening and Coloring Batch Transform for GANs

By Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe. 2018

56 citations recorded by the source corpus.

Batch Normalization (BN) is a common technique used to speed-up and stabilize training. On the other hand, the learnable parameters of BN are commonly used in conditional Generative Adversarial Networks (cGANs) for representing class-specific information using conditional Batch Normalization (cBN). In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization. We show that our conditional Coloring can represent categorical conditioning information which largely helps the cGAN qualitative results. Moreover, we show that full-feature whitening is important in a general GAN scenario in which the training process is known to be highly unstable. We test our approach on different datasets and using different GAN networks and training protocols, showing a consistent improvement in all the tested frameworks. Our CIFAR-10 conditioned results are higher than all previous works on this dataset.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Nicu Sebe
2. Aliaksandr Siarohin
3. Yue Wang
4. Enver Sangineto
5. Zhi Chen
6. Cynthia Rudin
7. Hang Zhao
8. Yijie Bei

In Statistical and Nonlinear Physics

1. Zhi Chen
2. Yijie Bei
3. Cynthia Rudin

Citations in the corpus

• Concept whitening for interpretable image recognition (2020, methodology): “Similarly, others have developed a differentiable whitening block based on Cholesky whitening (Siarohin et al., 2018).”
• Whitening for Self-Supervised Representation Learning (2020, methodology): “n more detail, given a set of samples V = fv ig, corresponding to the current mini-batch of images B= fx ig, we rst project the elements of V onto a spherical distribution using a whitening transform [38]. The whitened representations fz ig, corresponding to V, are used to compute a standard Mean Sq”
• On Feature Decorrelation in Self-Supervised Learning (2021, background): “Meanwhile, feature decorrelation appears to be a valuable idea in the field of machine learning: In discriminative tasks, [12, 48] introduce in objective functions additional terms regularizing correlation matrices and [25, 26] develop normalization layers standardizing covariance matrices to obtain”
• ContraGAN: Contrastive Learning for Conditional Image Generation (2020, background): “On this journey, conditioning class information for the generator and discriminator turns out to be the secret behind realistic image generation [17, 19, 20].”
• RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening (2021, background): “However, a number of studies [13, 14, 30, 56, 51, 36, 7, 45, 57] claim that feature covariance contains domain-specific style such as texture and color.”