Learning to Manipulate Individual Objects in an Image

By Yanchao Yang, Yutong Chen, Stefano Soatto. 2020

38 citations recorded by the source corpus.

We describe a method to train a generative model with latent factors that are (approximately) independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations, or any form of annotation for that matter. The key to our method is the combination of spatial disentanglement, enforced by a Contextual Information Separation loss, and perceptual cycle-consistency, enforced by a loss that penalizes changes in the image partition in response to perturbations of the latent factors. We test our method's ability to allow independent control of spatial and semantic factors of variability on existing datasets and also introduce two new ones that highlight the limitations of current methods.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, background): “Utilizing different perceptual [49, 80] or contrastive losses [46] could help overcome this limitation.”
On the Binding Problem in Artificial Neural Networks (2020, background): “, 2019), or an adversarial formulation (Yang et al., 2020).”
BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images (2020, background): “Recently, unsupervised methods [2, 12, 13, 26, 50, 55] learned object disentanglement for multi-object scenes on simpler synthetic datasets (single-colour objects, simple lighting, and material).”