FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

By Krishna Kumar Singh, Utkarsh Ojha, Yong Jae Lee. 2018

141 citations recorded by the source corpus.

We propose FineGAN, a novel unsupervised GAN framework, which disentangles the background, object shape, and object appearance to hierarchically generate images of fine-grained object categories. To disentangle the factors without supervision, our key idea is to use information theory to associate each factor to a latent code, and to condition the relationships between the codes in a specific way to induce the desired hierarchy. Through extensive experiments, we show that FineGAN achieves the desired disentanglement to generate realistic and diverse images belonging to fine-grained classes of birds, dogs, and cars. Using FineGAN's automatically learned features, we also cluster real images as a first attempt at solving the novel problem of unsupervised fine-grained object category discovery. Our code/models/demo can be found at https://github.com/kkanshul/finegan

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Object-Centric Image Generation from Layouts (2020, methodology): “Orthogonal to considerations relating to the input, methods tend to rely on additional components to help with the complexity of scene gen-Object-Centric eration, such as attention mechanisms [60,27] and explicit disentanglement of objects from the background [52].”
Unsupervised Discovery of Disentangled Manifolds in GANs (2020, methodology): “The FineGAN [10] and VON [11] model uses a hierarchy architecture to encode different types of variation into separated latent spaces.”
Contrastive Learning for Diverse Disentangled Foreground Generation (2022, background): “A large number of work try to disentangle object shape/structure from appearance [39, 27, 37, 7, 49].”
MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation (2019, methodology): “Our most related work is FineGAN [36], which leverages information theory [4] to disentangle background, object pose, shape, and texture with minimal supervision.”
Collaging Class-specific GANs for Semantic Image Synthesis (2021, background): “Others [38, 16, 26, 11, 15, 33, 35] model an image via different components, e.g., by having global and local branches [16, 26, 11, 15] with the local branch being specialized for different objects/parts.”
Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification (2019, background): “Recently, considerable attention has been focused on learning disentangled representations in various fields [12, 7, 13, 24].”