Labels4Free: Unsupervised Segmentation using StyleGAN
By Rameen Abdal, Peihao Zhu, Niloy J. Mitra, Peter Wonka. 2021
88 citations recorded by the source corpus.
We propose an unsupervised segmentation framework for StyleGAN generated objects. We build on two main observations. First, the features generated by StyleGAN hold valuable information that can be utilized towards training segmentation networks. Second, the foreground and background can often be treated to be largely independent and be swapped across images to produce plausible composited images. For our solution, we propose to augment the StyleGAN2 generator architecture with a segmentation branch and to split the generator into a foreground and background network. This enables us to generate soft segmentation masks for the foreground object in an unsupervised fashion. On multiple object classes, we report comparable results against state-of-the-art supervised segmentation networks, while against the best unsupervised segmentation approach we demonstrate a clear improvement, both in qualitative and quantitative metricsProject Page : https:/rameenabdal.github.io/Labels4Free
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Peihao Zhu
- Peter Wonka
- Rameen Abdal
- John C. Femiani
- Supasorn Suwajanakorn
- Eli Shechtman
- Daniel Cohen-Or
- Pitchaporn Rewatbowornwong
Citations in the corpus
- Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks (2021, methodology): “…that the GAN can be exploited for downstream tasks like unsupervised and few-shot part segmentation (Zhang et al., 2021; Tritrong et al., 2021; Abdal et al., 2021a; Collins et al., 2020; Bielski & Favaro, 2019), extracting 3D models of the objects (Pan et al., 2021; Chan et al., 2020) and…”
- Repurposing GANs for One-Shot Semantic Part Segmentation (2021, background): “Labels4Free [56] trains an unsupervised background/foreground separation network with GAN’s features.”
- Bridging CLIP and StyleGAN through Latent Alignment for Image Editing (2022, unknown): “And the generative prior of StyleGAN has been utilized in many downstream tasks [1, 3, 8, 24, 30].”