Improving Shape Deformation in Unsupervised Image-to-Image Translation

By Aaron Gokaslan, Vivek Ramanujan, Daniel Ritchie, Kwang In Kim, James Tompkin. 2018

78 citations recorded by the source corpus.

Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change. Inspired by semantic segmentation, we introduce a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator. This is coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects. We demonstrate that this design is more capable of representing shape deformation in a challenging toy dataset, plus in complex mappings with significant dataset variation between humans, dolls, and anime faces, and between cats and dogs.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Trevor Darrell
3. Ming-Hsuan Yang
4. Dacheng Tao
5. Alexei A. Efros
6. Bernt Schiele
7. Andrea Vedaldi
8. Xiaogang Wang

In Generative Models

1. Richard Zhang
2. Jun-Yan Zhu
3. Alexei A. Efros
4. Taesung Park
5. Eli Shechtman
6. Tao Qin
7. Jinwoo Shin
8. Zhibo Chen

Citations in the corpus

• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness (2018, background): “, 2017) all produce excellent results using standard CNNs, while CNNbased shape transfer seems to be very difficult (Gokaslan et al., 2018).”
• Contrastive Learning for Unpaired Image-to-Image Translation (2020, background): “Recent works further enable multiple domains and multimodal synthesis [9,83,1,38,42] and improve the quality of results [66,81,19,72,40].”
• Image-to-Image Translation: Methods and Applications (2021, methodology): “Using the I2I methods for artistic creation can directly translate real-world photographic works into illustrations in children’s books [226], cartoon images [130], [102], [36], [132], [134], [140], [32],”
• InstaGAN: Instance-aware Image-to-Image Translation (2018, methodology): “A few number of recent methods (Kim et al., 2017; Liu et al., 2017; Gokaslan et al., 2018) show some transfiguration results but only for images with a single instance often in a clear background.”