Drawing and Recognizing Chinese Characters with Recurrent Neural Network

By Xu-Yao Zhang, Fei Yin, Yanming Zhang, Chenglin Liu, Yoshua Bengio. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016

349 citations recorded by the source corpus.

Recent deep learning based approaches have achieved great success on handwriting recognition. Chinese characters are among the most widely adopted writing systems in the world. Previous research has mainly focused on recognizing handwritten Chinese characters. However, recognition is only one aspect for understanding a language, another challenging and interesting task is to teach a machine to automatically write (pictographic) Chinese characters. In this paper, we propose a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generative model for drawing (generating) Chinese characters. To recognize Chinese characters, previous methods usually adopt the convolutional neural network (CNN) models which require transforming the online handwriting trajectory into image-like representations. Instead, our RNN based approach is an end-to-end system which directly deals with the sequential structure and does not require any domain-specific knowledge. With the RNN system (combining an LSTM and GRU), state-of-the-art performance can be achieved on the ICDAR-2013 competition database. Furthermore, under the RNN framework, a conditional generative model with character embedding is proposed for automatically drawing recognizable Chinese characters. The generated characters (in vector format) are human-readable and also can be recognized by the discriminative RNN model with high accuracy. Experimental results verify the effectiveness of using RNNs as both generative and discriminative models for the tasks of drawing and recognizing Chinese characters.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/572b98c8d9b3f4f0078f8d1565ecfb9ccb3bfcab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Residual Recurrent Highway Networks for Learning Deep Sequence Prediction Models (2018, positive): “video interpretation [21, 27], language modeling [25, 28–30], speech recognition [31, 32], machine translation [33] and text classification [34, 35], etc.”
Object detection using YOLO: challenges, architectural successors, datasets and applications (2022, background): “The success stories of deep learning in various domains includes weather forecasting [80], stock market prediction [53], speech recognition [59], object detection [37], character recognition [83], intrusion detection [32], automatic landslide detection [48], time”
A Neural Representation of Sketch Drawings (2017, methodology): “The approach developed in [7] and later followed by [26] was to use different weightings for each pen event when calculating the losses, such as a hand-tuned weighting of (1 , 10 , 100) .”
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction (2017, methodology): “Inspired by recent breakthroughs in sequence-to-sequence learning for language modeling [25], [27] and geometric path generation [9], [28], as well as unsupervised end-toend learning of complex conditional distributions [6], [7], we design our learning framework as a neural network Conditional Varia”
Deep Learning for Classical Japanese Literature (2018, methodology): “Previous work [7, 27] utilized MDN-RNN to generate stroke-based Chinese characters.”