Normalizing tweets with edit scripts and recurrent neural embeddings

By Grzegorz Chrupała. 2014

85 citations recorded by the source corpus.

Tweets often contain a large proportion of abbreviations, alternative spellings, novel words and other non-canonical language. These features are problematic for standard language analysis tools and it can be desirable to convert them to canonical form. We propose a novel text normalization model based on learning edit operations from labeled data while incorporating features induced from unlabeled data via character-level neural text embeddings. The text embeddings are generated using an Simple Recurrent Network. We find that enriching the feature set with text embeddings substantially lowers word error rates on an English tweet normalization dataset. Our model improves on stateof-the-art with little training data and without any lexical resources.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Character-based Neural Machine Translation (2016, background): “…been explored in Natural Language Processing (NLP), e.g. for POS tagging (Santos and Zadrozny, 2014), name entity recognition (Santos and aes, 2015), parsing (Ballesteros et al., 2015), normalization (Chrupala, 2014) or learning word representations (Botha and Blunsom, 2014; Chen et al., 2015).”
A Primer on Neural Network Models for Natural Language Processing (2015, neutral): “, Bougares, Schwenk, & Bengio, 2014b), dependency parsing (Dyer et al., 2015; Watanabe & Sumita, 2015), sentiment analysis (Wang, Liu, SUN, Wang, & Wang, 2015b), noisy text normalization (Chrupala, 2014), dialog state tracking (Mrksic, O Seaghdha, Thomson, Gasic, Su, Vandyke, Wen, & ”
Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs (2015, background): “For instance, Chrupała (2014) used character-based recurrent neural networks to normalize tweets.”
RNN Approaches to Text Normalization: A Challenge (2016, background): “Some previous work, also on social media normalization, that has made use of neural techniques includes (Chrupała, 2014; Min and Mott, 2015).”
Robsut Wrod Reocginiton via Semi-Character Recurrent Neural Network (2016, methodology): “The character-based neural networks have been investigated and used for a variety of NLP tasks such as segmentation (Chru-pala 2013), dependency parsing (Ballesteros, Dyer, and Smith 2015), machine translation (Ling et al. 2015), and text normalization (Chrupała 2014).”
Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory (2015, methodology): “Recent works used RNNs to predict words or characters in a sequence (Chrupała, 2014; Zhang and Lapata, 2014).”