StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis

By Peter Schaldenbrand, Zhixuan Liu, Jean Oh. 2021

23 citations recorded by the source corpus.

Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated. We introduce StyleCLIPDraw which adds a style loss to the CLIPDraw text-to-drawing synthesis model to allow artistic control of the synthesized drawings in addition to control of the content via text. Whereas performing decoupled style transfer on a generated image only affects the texture, our proposed coupled approach is able to capture a style in both texture and shape, suggesting that the style of the drawing is coupled with the drawing process itself. More results and our code are available at https://github.com/pschaldenbrand/StyleCLIPDraw

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Control and Systems Engineering, Generative Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Generative Models

In Multimodal Learning

Citations in the corpus

Zero-Shot Text-Guided Object Generation with Dream Fields (2021, background): “Some works have optimized parameters of vector graphics, suggesting CLIP guidance is highly general [17, 23, 50].”