Adversarial Generation of Handwritten Text Images Conditioned on Sequences

By Eloi Alonso, Bastien Moysset, Ronaldo O. Messina. 2019

103 citations recorded by the source corpus.

We propose a system based on Generative Adversarial Networks (GAN) to produce synthetic images of handwritten words. We use bidirectional LSTM recurrent layers to get an embedding of the word to be rendered, and we feed it to the generator network. We also modify the standard GAN by adding an auxiliary network for text recognition. The system is then trained with a balanced combination of an adversarial loss and a CTC loss. Together, these extensions to GAN enable to control the textual content of the generated word images yielding realistic-looking images on both French and Arabic languages. State-of-the-art offline handwriting text recognition systems tend to use neural networks and therefore require a large amount of annotated data to be trained. In order to partially satisfy this requirement, we could use those synthetic images to increase the amount of training data. We show that integrating generated images into the existing training data of a text recognition system can slightly enhance its performance.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Handwriting Transformers (2021, methodology): “However, both [3, 7] do not adapt to a specific author’s writing style.”
Synthetic Data for Deep Learning (2019, background): “[14] consider adversarial generation of handwritten text (see also Section 2.”
Diffusion models for Handwriting Generation (2020, methodology): “GAN based methods for handwriting generation (Alonso et al., 2019; Kang et al., 2020; Fogel et al., 2020) have been used as an alternative to online autoregressive models.”