Boosting Named Entity Recognition with Neural Character Embeddings

By C. D. Santos, Victor Guimarães. 2015

344 citations recorded by the source corpus.

Most state-of-the-art named entity recognition (NER) systems rely on handcrafted features and on the output of other NLP tasks such as part-of-speech (POS) tagging and text chunking. In this work we propose a language-independent NER system that uses automatically learned features only. Our approach is based on the CharWNN deep neural network, which uses word-level and character-level representations (embeddings) to perform sequential classification. We perform an extensive number of experiments using two annotated corpora in two different languages: HAREM I corpus, which contains texts in Portuguese; and the SPA CoNLL-2002 corpus, which contains texts in Spanish. Our experimental results shade light on the contribution of neural character embeddings for NER. Moreover, we demonstrate that the same neural network which has been successfully applied to POS tagging can also achieve state-of-the-art results for language-independet NER, using the same hyperparameters, and without any handcrafted features. For the HAREM I corpus, CharWNN outperforms the state-of-the-art system by 7.9 points in the F1-score for the total scenario (ten NE classes), and by 7.2 points in the F1 for the selective scenario (five NE classes).

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “[30] applied character-level representations, along with word embeddings for NER, achieving state-of-the-art results in Portuguese and Spanish corpora.”
Character-based Neural Machine Translation (2016, background): “for POS tagging (Santos and Zadrozny, 2014), name entity recognition (Santos and aes, 2015), parsing (Ballesteros et al.”
Named Entity Recognition with Bidirectional LSTM-CNNs (2015, methodology): “To induce character-level features, we use a convolutional neural network, which has been successfully applied to Spanish and Portuguese NER (Santos et al., 2015) and German POS-tagging (Labeau et al.”
Charagram: Embedding Words and Sentences via Character n-grams (2016, background): “…word types for several tasks, including language modeling (Kim et al., 2015), part-of-speech tagging (dos Santos and Zadrozny, 2014), named entity recognition (dos Santos and Guimar˜aes, 2015), text classiﬁcation (Zhang et al., 2015), and machine translation (Costa-Juss`a and Fonollosa, 2016).”
Chinese NER Using Lattice LSTM (2018, methodology): “Collobert et al. (2011) used a CNN-CRF structure, obtaining competitive results to the best statistical models. dos Santos et al. (2015) used character CNN to augment a CNN-CRF model.”
Multi-Task Cross-Lingual Sequence Tagging from Scratch (2016, methodology): “Some multilingual taggers that do not rely on feature engineering have also been presented (Lam-ple et al., 2016; dos Santos et al., 2015), but while these methods are language-independent, they do not study the effect of cross-lingual joint training.”
Portuguese Named Entity Recognition using BERT-CRF (2019, background): “F1 CharWNN (Santos and Guimaraes, 2015) 67.16 63.74 65.41 73.98 68.68 71.23 LSTM-CRF (Castro et al., 2018) 72.78 68.03 70.33 78.26 74.39 76.27 BiLSTM-CRF+FlairBBP (Santos et al., 2019a) 74.91 74.37 74.64 83.38 81.17 When evaluating, we produce valid predictions by removing all invalid tag transition”
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (2018, methodology): “Santos and Guimaraes (2015) implemented a model with a CNN over the characters of word, concatenated with word embeddings of the central word and its neighbors, fed to a feed forward network, and followed by the Viterbi algorithm to predict labels for each word.”
Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts (2017, background): “…is a recent surge of interests in incorporating word embeddings in a variety of NLP tasks such as Named Entity Recognition (Lample et al. 2016), (Santos and Guimaraes 2015), syntactic parsing (Bansal, Gimpel, and Livescu 2014), semantic relation extraction (Nguyen and Grishman 2015), (Fu et al.…”
Toward Mention Detection Robustness with Recurrent Neural Networks (2016, background): “…neural networks (CNNs) (Yih et al., 2014; Shen et al., 2014; Kalchbrenner et al., 2014; Kim, 2014; Zeng et al., 2014; dos Santos et al., 2015a; dos Santos and Guimarães, 2015b), recurrent/recursive neural networks (Socher et al., 2012; Cho et al., 2014a; Bahdanau et al., 2015; Zhou and Xu,…”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “Santos and Guimaraes [30] applied character-level representations, along with word embeddings for NER, achieving state-of-the-art results in Portuguese and Spanish corpora.”