ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis

By I-Chao Shen, Bing-Yu Chen. 2021

40 citations recorded by the source corpus.

This article presents a novel deep learning-based approach for automatically vectorizing and synthesizing the clipart of man-made objects. Given a raster clipart image and its corresponding object category (e.g., airplanes), the proposed method sequentially generates new layers, each of which is composed of a new closed path filled with a single color. The final result is obtained by compositing all layers together into a vector clipart image that falls into the target category. The proposed approach is based on an iterative generative model that (i) decides whether to continue synthesizing a new layer and (ii) determines the geometry and appearance of the new layer. We formulated a joint loss function for training our generative model, including the shape similarity, symmetry, and local curve smoothness losses, as well as vector graphics rendering accuracy loss for synthesizing clipart recognizable by humans. We also introduced a collection of man-made object clipart, ClipNet, which is composed of closed-path layers, and two designed preprocessing tasks to clean up and enrich the original raw clipart. To validate the proposed approach, we conducted several experiments and demonstrated its ability to vectorize and synthesize various clipart categories. We envision that our generative model can facilitate efficient and intuitive clipart designs for novice users and graphic designers.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

1. Kevin Frans
2. Olaf Witkowski
3. Lisa B. Soros

In Computer Vision and Pattern Recognition

1. Kevin Frans
2. Olaf Witkowski
3. Lisa B. Soros

In Generative Models

1. Kevin Frans
2. Olaf Witkowski
3. Lisa B. Soros

Citations in the corpus

• CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “Image generation methods that operate over vector images have traditionally required a vector-based dataset, however recent work has shown how differentiable renderers can be used to bypass this limitation (Reddy et al., 2021; Shen & Chen, 2021).”