Neural Networks for Text Correction and Completion in Keyboard Decoding

By Shaon Ghosh, Per Ola Kristensson. 2017

76 citations recorded by the source corpus.

Despite the ubiquity of mobile and wearable text messaging applications, the problem of keyboard text decoding is not tackled sufficiently in the light of the enormous success of the deep learning Recurrent Neural Network (RNN) and Convolutional Neural Networks (CNN) for natural language understanding. In particular, considering that the keyboard decoders should operate on devices with memory and processor resource constraints, makes it challenging to deploy industrial scale deep neural network (DNN) models. This paper proposes a sequence-to-sequence neural attention network system for automatic text correction and completion. Given an erroneous sequence, our model encodes character level hidden representations and then decodes the revised sequence thus enabling auto-correction and completion. We achieve this by a combination of character level CNN and gated recurrent unit (GRU) encoder along with and a word level gated recurrent unit (GRU) attention decoder. Unlike traditional language models that learn from billions of words, our corpus size is only 12 million words; an order of magnitude smaller. The memory footprint of our learnt model for inference and prediction is also an order of magnitude smaller than the conventional language model based text decoders. We report baseline performance for neural keyboard decoders in such limited domain. Our models achieve a word level accuracy of $90\%$ and a character error rate CER of $2.4\%$ over the Twitter typo dataset. We present a novel dataset of noisy to corrected mappings by inducing the noise distribution from the Twitter data over the OpenSubtitles 2009 dataset; on which our model predicts with a word level accuracy of $98\%$ and sequence accuracy of $68.9\%$. In our user study, our model achieved an average CER of $2.6\%$ with the state-of-the-art non-neural touch-screen keyboard decoder at CER of $1.6\%$.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Federated Learning for Mobile Keyboard Prediction (2018, methodology): “RNN language models optimized for word prediction rate and keystroke savings within inference-time latency and memory constraints have also been published [14, 15].”
Foreign English Accent Adjustment by Learning Phonetic Patterns (2018, methodology): “Complex models such as sequence-to-sequence recurrent neural networks (seq2seq RNN) have shown good results on the language processing and modification problems (Cho et al., 2014; Ghosh and Kristensson, 2017).”
Accelerating Text Communication via Abbreviated Sentence Input (2021, unknown): “Ghosh and Kristensson (2017) corrected typos in tweets to a low character error rate of 2.4% by using a character convolutional neural network, an encoder with gated recurrent units, and a decoder with attention.”