CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

By Yang Chen, Yu-Kun Lai, Yong-jin Liu. 2018

402 citations recorded by the source corpus.

In this paper, we propose a solution to transforming photos of real-world scenes into cartoon style images, which is valuable and challenging in computer vision and computer graphics. Our solution belongs to learning based methods, which have recently become popular to stylize images in artistic forms such as painting. However, existing methods do not produce satisfactory results for cartoonization, due to the fact that (1) cartoon styles have unique characteristics with high level simplification and abstraction, and (2) cartoon images tend to have clear edges, smooth color shading and relatively simple textures, which exhibit significant challenges for texture-descriptor-based loss functions used in existing methods. In this paper, we propose CartoonGAN, a generative adversarial network (GAN) framework for cartoon stylization. Our method takes unpaired photos and cartoon images for training, which is easy to use. Two novel losses suitable for cartoonization are proposed: (1) a semantic content loss, which is formulated as a sparse regularization in the high-level feature maps of the VGG network to cope with substantial style variation between photos and cartoons, and (2) an edge-promoting adversarial loss for preserving clear edges. We further introduce an initialization phase, to improve the convergence of the network to the target manifold. Our method is also much more efficient to train than existing methods. Experimental results show that our method is able to generate high-quality cartoon images from real-world photos (i.e., following specific artists' styles and with clear edges and smooth shading) and outperforms state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, positive): “Already by now, NAS methods have outperformed manually designed architectures on a range of tasks such as image classification (Tan & Le, 2019; Tan et al., 2019; Howard et al., 2019; Liu et al., 2018a), and segmentation (Chen et al., 2018a; Liu et al., 2019; Chen et al., 2019b).”
Applications of Generative Adversarial Networks (GANs): An Updated Review (2019, background): “…could treat a certain disease, Object detection [28, 66, 66, 148], retrieving images that contain a given pattern [6, 76, 109, 112], Facial Attribute Manipulation [52, 59, 125, 137, 141], Anime Character Generation [18, 49],Image to Image Translation [15, 45, 47, 74, 104, 135, 147] and many more.”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “This problem can be alleviated by resorting to an image-to-image framework (Zhu et al. 2017; Chen, Lai, and Liu 2018; Chen, Liu, and Chen 2020).”
Neural Style Transfer: A Review (2017, background): “[95] propose a CartoonGAN to automatically transform real-word photos to”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “GANs have also been utilized in other image processing and computer vision tasks [246], [247], [248], such as object transfiguration [249], [250], semantic segmentation”
Diversify and Match: A Domain Adaptive Representation Learning Paradigm for Object Detection (2019, methodology): “To verify the universality of Domain Diversification (DD) for the domain shift architecture, we conducted additional experiments on the domain shifter from CartoonGAN [1].”
Positional Normalization (2019, background): “…input image in the target domain, 3) LPIPS cont : average LPIPS distance between each output image and its input in the source domain, and 4) perceptual loss (VGG) [32, 62]: L1 distance between the VGG-19 Conv4_4 features [8] of each output image and its corresponding input in the source domain.”
Application of deep learning algorithms in geotechnical engineering: a short critical review (2021, background): “…as natural synthesis (Brock et al. 2018; Karras et al. 2017, 2019; Salimans et al. 2016), face synthesis (Bao et al. 2018), image style transfer (Chen et al. 2018a; Choi et al. 2018; Ma et al. 2018b; Taigman et al. 2016), image segmentation (Zhang et al. 2018c), facial expression recognition…”
Self-Supervised CycleGAN for Object-Preserving Image-to-Image Domain Adaptation (2020, background): “Witnessing the success of cycle-consistency-based approaches [36,13,32], an increasing number of researchers [4,6,22,25] made their eﬀort to the area of un-paired I2I translation.”
Image-to-Image Translation: Methods and Applications (2021, background): “[37], [35], [34], [76], [152], [81], comics [33], [190] or a multichirography of Chinese characters [191].”
APDrawingGAN: Generating Artistic Portrait Drawings From Face Photos With Hierarchical GANs (2019, background): “, [15, 40, 2, 5]) have achieved especially good results, by utilizing sets of (paired or unpaired) photos and stylized images for learning.”
Controllable Artistic Text Style Transfer via Shape-Matching GAN (2019, background): “Meanwhile, other researchers regard style transfer as an image-to-image translation problem [16, 32], and exploited Generative Adversarial Network (GAN) [12] to transfer specialized styles such as cartoons [7], paintings [25] and makeups [8, 5].”
U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation (2019, background): “CartoonGAN, as shown in the supplementary materials, has only changed the over-all color of the image to an animated style, but compared to selﬁe, the eye, which is the biggest characteristic of animation, has not changed at all.”
Sphere Generative Adversarial Network Based on Geometric Moment Matching (2019, methodology): “[8], generative adversarial networks (GANs) have attracted much research interest, and they have been used to achieve outstanding performance in a wide range of computer vision applications including in image generation [17, 26], super resolution [14], video prediction [19], style transfer [5, 12, 3”