Named Entity Recognition with Bidirectional LSTM-CNNs

By Jason P. C. Chiu, Eric Nichols. 2015

1,992 citations recorded by the source corpus.

Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineering and lexicons to achieve high performance. In this paper, we present a novel neural network architecture that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering. We also propose a novel method of encoding partial lexicon matches in neural networks and compare it to existing approaches. Extensive evaluation shows that, given only tokenized text and publicly available word embeddings, our system is competitive on the CoNLL-2003 dataset and surpasses the previously reported state of the art performance on the OntoNotes 5.0 dataset by 2.13 F1 points. By using two lexicons constructed from publicly-available sources, we establish new state of the art performance with an F1 score of 91.62 on CoNLL-2003 and 86.28 on OntoNotes, surpassing systems that employ heavy feature engineering, proprietary lexicons, and rich entity linking information.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semi-Supervised Sequence Modeling with Cross-View Training (2018, methodology): “Our models use two layer CNN-BiLSTM encoders (Chiu and Nichols, 2016; Ma and Hovy, 2016; Lample et al., 2016) and task-speciﬁc prediction modules.”
Semi-supervised sequence tagging with bidirectional language models (2017, methodology): “The baseline sequence tagger uses 30 dimensional character embeddings and a CNN with 30 ﬁlters of width 3 characters followed by a tanh non-linearity for the token character encoder.”
Recent Trends in Deep Learning Based Natural Language Processing (2017, neutral): “[141] concatenated lexicon features, character embeddings and word embeddings and fed them as input to a bidirectional LSTM.”
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (2016, methodology): “Among these neural architectures, the three approaches most similar to our model are the BLSTM-CRF model proposed by Huang et al. (2015), the LSTM-CNNs model by Chiu and Nichols (2015) and the BLSTM-CRF by Lample et al. (2016).”
Word Representation Models for Morphologically Rich Languages in Neural Machine Translation (2016, background): “Several authors have proposed convolutional neural networks over character sequences, as part of neural models for part of speech tagging (Santos and Zadrozny, 2014), named entity recognition (Ma and Hovy, 2016; Chiu and Nichols, 2015), language modelling (Kim et al.”
Assessing The Factual Accuracy of Generated Text (2019, background): “Entities can be identified with named-entity recognition (NER) systems like Chiu and Nichols [5], Finkel et al. [7], Lample et al. [9] that take in 𝑋 and produce the set { 𝑒 } .”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, methodology): “In accordance with previous work (Chiu and Nichols 2016; Lample et al. 2016) the tagging module contains an RNN-based encoding layer followed by a sequence tagging module based on a conditional random field (Lafferty, McCallum, and Pereira 2001).”
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (2018, background): “Many attempts have been made to conduct end-to-end learning and build reliable models without handcrafted features (Chiu and Nichols, 2016; Lample et al., 2016; Ma and Hovy, 2016).”
Transfer Learning for Sequence Labeling Using Source Model and Target Data (2019, methodology): “All models are implemented in TensorFlow (Abadi et al. 2015), as an extension of NeuroNER (Dernoncourt, Lee, and Szolovits 2017).”
Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme (2017, methodology): “Recently, several neural network architectures (Chiu and Nichols, 2015; Huang et al., 2015; Lample et al., 2016) have been successfully applied to NER, which is regarded as a sequential token tagging task.”
SpanNER: Named Entity Re-/Recognition as Span Prediction (2021, methodology): “NER as Different Tasks Although NER is commonly formulated as a sequence labeling task (Chiu and Nichols, 2015; Huang et al., 2015; Ma and Hovy, 2016; Lample et al., 2016; Akbik et al., 2018; Peters et al., 2018; Devlin et al., 2018; Xia et al., 2019; Akbik et al., 2019; Luo et al., 2020; Lin et al.”
Leveraging Knowledge Bases in LSTMs for Improving Machine Reading (2017, methodology): “At each time step, our models take as input a word vector and a capitalization feature (Chiu and Nichols, 2016).”
Deep Neural Architectures for Joint Named Entity Recognition and Disambiguation (2019, neutral): “[6] use a sequential bidirectional LSTMCNN.”