Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks

By Peihao Zhu, Rameen Abdal, John C. Femiani, Peter Wonka. 2021

94 citations recorded by the source corpus.

We present a new method for one shot domain adaptation. The input to our method is trained GAN that can produce images in domain A and a single reference image I_B from domain B. The proposed algorithm can translate any output of the trained GAN from domain A to domain B. There are two main advantages of our method compared to the current state of the art: First, our solution achieves higher visual quality, e.g. by noticeably reducing overfitting. Second, our solution allows for more degrees of freedom to control the domain gap, i.e. what aspects of image I_B are used to define the domain B. Technically, we realize the new method by building on a pre-trained StyleGAN generator as GAN and a pre-trained CLIP model for representing the domain gap. We propose several new regularizers for controlling the domain gap to optimize the weights of the pre-trained StyleGAN generator to output images in domain B instead of domain A. The regularizers prevent the optimization from taking on too many attributes of the single reference image. Our results show significant visual improvements over the state of the art as well as multiple applications that highlight improved control.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

GAN Inversion: A Survey (2021, background): “few target images [120] (few-shot and one-shot), pretrained language-image models [121] (zero-shot), or both [122].”
One-Shot Generative Domain Adaptation (2021, unknown): “In particular, the diversity-truncation strategy could also help the original FreezeD slightly, shown in the “FD-trunc.” column of the above table.”
Third Time's the Charm? Image and Video Editing with StyleGAN3 (2022, background): “Multiple works leverage the ﬁne-tuning of StyleGAN2 generators for various applications, such as image-to-image translation [21,52,64,74,84] and inversion [7,10,56].”