Controllable Artistic Text Style Transfer via Shape-Matching GAN

By Shuai Yang, Zhangyang Wang, Zhaowen Wang, Ning Xu, Jiaying Liu, Zongming Guo. 2019

111 citations recorded by the source corpus.

Artistic text style transfer is the task of migrating the style from a source image to the target text to create artistic typography. Recent style transfer methods have considered texture control to enhance usability. However, controlling the stylistic degree in terms of shape deformation remains an important open challenge. In this paper, we present the first text style transfer network that allows for real-time control of the crucial stylistic degree of the glyph through an adjustable parameter. Our key contribution is a novel bidirectional shape matching framework to establish an effective glyph-style mapping at various deformation levels without paired ground truth. Based on this idea, we propose a scale-controllable module to empower a single network to continuously characterize the multi-scale shape features of the style image and transfer these features to the target text. The proposed method demonstrates its superiority over previous state-of-the-arts in generating diverse, controllable and high-quality stylized text.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Zhangyang Wang
2. Zongming Guo
3. Jiaying Liu

In Computer Graphics and Computer-Aided Design

1. Zhangyang Wang
2. Xinyu Gong
3. Shuai Yang
4. Shiyu Chang
5. Zongming Guo
6. Jiaying Liu

In Computer Vision and Pattern Recognition

1. Zhangyang Wang
2. Shiyu Chang
3. Jiaying Liu
4. Xinyu Gong
5. Zongming Guo
6. Shuai Yang
7. Chen Change Loy
8. Wenhan Yang

Citations in the corpus

• AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “These encoder-decoder GANs also gain popularity in image enhancement (Wang et al., 2018a; Kupyn et al., 2019; Jiang et al., 2019) and editing (Sanakoyeu et al., 2018; Yang et al., 2019).”
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “It has achieved great success in many generation and synthesis tasks, such as text-to-image translation [66, 65, 61, 48], image-to-image translation [22, 70, 63], and image enhancement [33, 31, 23].”
• SEAN: Image Synthesis With Semantic Region-Adaptive Normalization (2019, unknown): “In most existing methods, styles are encoded in three places: i) statistics of image features [13, 43]; ii) neural network weights ( e.g . fast style transfer [24, 54, 50]); iii) parameters of a network normalization layer [21, 28]).”
• TransGAN: Two Transformers Can Make One Strong GAN (2021, background): “…on numerous tasks including image synthesis (Radford et al., 2015; Arjovsky et al., 2017; Gulrajani et al., 2017; Miyato et al., 2018; Brock et al., 2018), image translation (Isola et al., 2017; Zhu et al., 2017a;b; Chen et al., 2020b), and image editing (Yang et al., 2019; Jiang et al., 2021).”
• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up (2021, background): “1 Introduction Generative adversarial networks (GANs) have gained considerable success on numerous tasks [1, 2, 3, 4, 5, 6, 7].”
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches (2020, methodology): “As we will show later, conditioning by label concatenation or feature interpolation [34] fails to properly condition G about the refinement level.”