Multi-content GAN for Few-Shot Font Style Transfer

By Samaneh Azadi, Matthew Fisher, Vladimir G. Kim, Zhaowen Wang, Eli Shechtman, Trevor Darrell. 2017

350 citations recorded by the source corpus.

In this work, we focus on the challenge of taking partial observations of highly-stylized text and generalizing the observations to generate unobserved glyphs in the ornamented typeface. To generate a set of multi-content images following a consistent style from very few examples, we propose an end-to-end stacked conditional GAN model considering content along channels and style along network layers. Our proposed network transfers the style of given glyphs to the contents of unseen ones, capturing highly stylized fonts found in the real-world such as those on movie posters or infographics. We seek to transfer both the typographic stylization (ex. serifs and ears) as well as the textual stylization (ex. color gradients and effects.) We base our experiments on our collected data set including 10,000 fonts with different styles and demonstrate effective generalization from a very small number of observed glyphs.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Voice Cloning with a Few Samples (2018, background): “…models based on deep neural networks have been successfully applied to many domains such as image generation [e.g., Oord et al., 2016b, Karras et al., 2017], speech synthesis [e.g., Oord et al., 2016a, Arik et al., 2017a, Wang et al., 2017], and language modeling [e.g., Jozefowicz et al., 2016].”
Discriminator Rejection Sampling (2018, methodology): “They have also been applied successfully to semi-supervised and unsupervised learning (Springenberg, 2015; Odena, 2016; Kumar et al., 2017), image editing (Yu et al., 2018; Ledig et al., 2017), and image style transfer (Zhu et al., 2017; Isola et al., 2017; Yi et al., 2017; Azadi et al., 2018).”
Generalizing from a Few Examples (2019, methodology): “, is presented in [131]. Besides character recognition and image classification, other image applications have also been considered. These include object recognition [35, 36, 82], font style transfer [7], phrase grounding [162], image retrieval [130], object tracking [14], specific object counting i”
APDrawingGAN: Generating Artistic Portrait Drawings From Face Photos With Hierarchical GANs (2019, background): “, [15, 40, 2, 5]) have achieved especially good results, by utilizing sets of (paired or unpaired) photos and stylized images for learning.”
Compositional GAN: Learning Conditional Image Composition (2018, background): “However, since generative models cannot generalize very well to a new example, we continue optimizing network parameters given the two input test instances to remove artifacts and generate sharper results [2].”
Compositional GAN: Learning Image-Conditional Binary Composition (2018, background): “The input cue could be in the form of an image (Isola et al. 2017; Zhu et al. 2017a; Liu et al. 2017; Azadi et al. 2017; Wang et al. 2017; Pathak et al. 2016), a text phrase (Zhang et al. 2017; Reed et al. 2016b, a; Johnson et al. 2018) or a class label layout (Mirza and Osindero 2014; Odena et al.…”
Disentangled representation learning in cardiac image analysis (2019, neutral): “Our approach here can be seen as similar to a disentanglement of an image into “style” and “content” (Gatys et al., 2016; Azadi et al., 2018), where we represent content (i.”
Differentiable vector graphics rasterization for editing and learning (2020, background): “Azadi et al. [2018] and Yue et al. [2019] generate new fonts given a few example styles.”
More Control for Free! Image Synthesis with Semantic Diffusion Guidance (2021, neutral): “Current image-conditioned synthesis techniques either only transfer the “style” of a reference image to a target image [21, 3] or are restricted to the domains with well-defined structure, e.”
Im2Vec: Synthesizing Vector Graphics without Vector Supervision (2021, background): “Prior works [2, 12] can generate new font glyphs from partial observations, but they only work in a low-resolution raster domain.”
Controllable Artistic Text Style Transfer via Shape-Matching GAN (2019, methodology): “[1] trained an MC-GAN for fast text style transfer, which, however, can only render 26 capital letters.”
Semantic Bottleneck Scene Generation (2019, background): “In conditional image synthesis one aims to generate images by conditioning on an input which can be provided in the form of an image [18, 47, 3, 5, 25], a text phrase [37, 45, 35, 2, 17], a scene graph [20, 2], a class label or a semantic lay-out [31, 8, 39, 32].”
Benchmark for Compositional Text-to-Image Synthesis (2021, background): “Another line of work explores disentanglement of different factors for image synthesis [2, 25, 14, 43].”
Language-Driven Image Style Transfer (2021, background): “By contrast, artistic style transfer [63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 2, 3, 83, 84, 85, 86] captures style concepts from reference and modiﬁes color distributions and texture patterns of content images.”
A Survey on Few-Shot Class-Incremental Learning (2023, background): “As a result, it has broad applications in zero-shot and few-shot domains, including style transfer, feature space transfer for data augmentation, and label-e ffi cient learning of transferable representations across domains (Azadi et al., 2018; Liu et al., 2018; Luo et al., 2017).”
Neural Style Transfer: A Review (2017, background): “al Network (GAN), to some extent, the training process is not that stable. But we believe that GAN-based style transfer is a very promising direction, and there are already some other GAN-based works [80,83,89,90] (Section5) in the ﬁeld of NST. 3) Results of MSPM Fast Style Transfer. The third block”
Deep Learning for Text Style Transfer: A Survey (2020, background): “2021), handwriting (Azadi et al. 2018; Zhang and Liu 2013), and videos (Ruder, Dosovitskiy, and Brox 2016; Chen et al.”