Few-shot Image Generation via Cross-domain Correspondence

By Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, Richard Zhang. 2021

318 citations recorded by the source corpus.

Training generative models, such as GANs, on a target domain containing limited examples (e.g., 10) can easily result in overfitting. In this work, we seek to utilize a large source domain for pretraining and transfer the diversity information from source to target. We propose to preserve the relative similarities and differences between instances in the source via a novel cross-domain distance consistency loss. To further reduce overfitting, we present an anchor-based strategy to encourage different levels of realism over different regions in the latent space. With extensive results in both photorealistic and non-photorealistic domains, we demonstrate qualitatively and quantitatively that our few-shot model automatically discovers correspondences between source and target domains and generates more diverse and realistic images than previous methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

StyleGAN-NADA (2021, methodology): “[35], Mine-GAN [55], TGAN [56] and TGAN + ADA [20].”
InstructPix2Pix: Learning to Follow Image Editing Instructions (2022, neutral): “Generative models for image editing Image editing models traditionally targeted a single editing task such as style transfer [15, 16] or translation between image domains [22, 24, 37, 43, 72].”
Ablating Concepts in Text-to-Image Diffusion Models (2023, background): “Fine-tuning aims to adapt the weights of a pretrained generative model to new domains [73, 46, 72, 41, 79, 34, 47, 80, 30, 35, 24, 44], downstream tasks [71, 54, 78], and test images [6, 53, 48, 31, 25, 49].”
LFS-GAN: Lifelong Few-Shot Image Generation (2023, methodology): “Our training configurations came from [6, 12].”
One-Shot Generative Domain Adaptation (2021, methodology): “Akin to Cross-domain [10], FID serves as the metric for evaluation.”
Few-Shot Learning With a Strong Teacher (2021, methodology): “Meta-learning based FSL has been applied in various ﬁelds, such as unsupervised learning [68], human motion prediction [20], domain adaptation [69], image generation [70], and object detection [71], [72].”
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing (2021, background): “Domain Adaptation Many works [18,46,51,68] have explored fine-tuning a pre-trained StyleGAN towards semantically similar domains.”
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks (2021, methodology): “The only other published method that accomplishes similar one-shot GAN domain adaptation which we are aware of is Ojha et al. (2021).”
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (2022, background): “[47] Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A Efros, Yong Jae Lee, Eli Shechtman, and Richard Zhang.”
Lafite2: Few-shot Text-to-Image Generation (2022, background): “In the literature, it is known that training and ﬁne-tuning large unconditional generative models with limited samples is not an easy task [13,22,42,43], not to mention that one needs to additionally consider image-text correspondence besides image quality.”
Modeling Image Composition for Complex Scene Generation (2022, background): “Recently, [7, 19, 25, 38] extend the few-shot generation to objects with similar structures, such as human faces, buildings and cars.”
StyleAlign: Analysis and Applications of Aligned StyleGAN Models (2021, background): “We next experiment with even farther domains, with barely any similarity between parent and child, such as human faces and churches, which were also examined by Ojha et al. (2021).”