Im2Vec: Synthesizing Vector Graphics without Vector Supervision

By Pradyumna Reddy, Michael Gharbi, Michal Lukác, Niloy J. Mitra. 2021

164 citations recorded by the source corpus.

Vector graphics are widely used to represent fonts, logos, digital artworks, and graphic designs. But, while a vast body of work has focused on generative algorithms for raster images, only a handful of options exists for vector graphics. One can always rasterize the input graphic and resort to image-based generative approaches, but this negates the advantages of the vector representation. The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time. This is not ideal because large-scale high- quality vector-graphics datasets are difficult to obtain. Furthermore, the vector representation for a given design is not unique, so models that supervise on the vector representation are unnecessarily constrained. Instead, we propose a new neural network that can generate complex vector graphics with varying topologies, and only requires indirect supervision from readily-available raster training images (i.e., with no vector counterparts). To enable this, we use a differentiable rasterization pipeline that renders the generated vector shapes and composites them together onto a raster canvas. We demonstrate our method on a range of datasets, and provide comparison with state-of-the-art SVG-VAE and DeepSVG, both of which require explicit vector graphics supervision. Finally, we also demonstrate our approach on the MNIST dataset, for which no groundtruth vector representation is available. Source code, datasets and more results are available at http://geometry.cs.ucl.ac.uk/projects/2021/Im2Vec/.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yue Wang
2. Yicheng Liu
3. Hang Zhao
4. Yilun Wang
5. Wotao Yin
6. Yin Zhang
7. David R Ha
8. Yingtao Tian

In Computer Graphics and Computer-Aided Design

1. Yue Wang
2. Kevin Frans
3. Olaf Witkowski
4. Yicheng Liu
5. Lisa B. Soros
6. Hang Zhao
7. Yilun Wang

In Computer Vision and Pattern Recognition

1. Yue Wang
2. Yilun Wang
3. Hang Zhao
4. Yicheng Liu
5. Kevin Frans
6. Olaf Witkowski
7. Lisa B. Soros
8. Luc Van Gool

In Generative Models

1. Yue Wang
2. Kevin Frans
3. Olaf Witkowski
4. Lisa B. Soros
5. Hang Zhao

Citations in the corpus

• CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “Image generation methods that operate over vector images have traditionally required a vector-based dataset, however recent work has shown how differentiable renderers can be used to bypass this limitation (Reddy et al., 2021; Shen & Chen, 2021).”
• VectorMapNet: End-to-end Vectorized HD Map Learning (2022, background): “Several recent works (Car-lier et al., 2020; Reddy et al., 2021) use different vector representations to generate vector images.”