Hierarchical Contextualized Representation for Named Entity Recognition

By Ying Luo, Fengshun Xiao, Zhao Hai. 2019

141 citations recorded by the source corpus.

Named entity recognition (NER) models are typically based on the architecture of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the modeling of single input prevent the full utilization of global information from larger scope, not only in the entire sentence, but also in the entire document (dataset). In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: sentence-level representation and document-level representation. In sentence-level, we take different contributions of words in a single sentence into consideration to enhance the sentence representation learned from an independent BiLSTM via label embedding attention mechanism. In document-level, the key-value memory network is adopted to record the document-aware information for each unique word which is sensitive to similarity of context information. Our two-level hierarchical contextualized representations are fused with each input token embedding and corresponding hidden state of BiLSTM, respectively. The experimental results on three benchmark NER datasets (CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset) show that we establish new state-of-the-art results.

Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pengfei Liu
2. Graham Neubig
3. Xuanjing Huang
4. Jinlan Fu
5. Yue Zhang
6. Leyang Cui
7. Xipeng Qiu
8. Yu Wu

In Natural Language Processing

1. Pengfei Liu
2. Xuanjing Huang
3. Jinlan Fu
4. Graham Neubig
5. Yue Zhang
6. Leyang Cui
7. Aixin Sun
8. Sen Yang

In Information Extraction

1. Pengfei Liu
2. Graham Neubig
3. Xuanjing Huang
4. Jinlan Fu
5. Yue Zhang
6. Yoav Goldberg

Citations in the corpus

• Interpretable Multi-dataset Evaluation for Named Entity Recognition (2020, background): “Orthogonally, some other works (Qian et al., 2018; Hu et al., 2020; Luo et al., 2020; Li et al., 2020; Lin et al., 2020) evaluate holistic metrics such as F1 across multiple datasets that differ in domain, language, or other characteristics (Sang and De Meulder, 2003; Collobert et al., 2011;…”
• A Survey on Deep Learning for Named Entity Recognition (2018, background): “07% [134] LSTM GloVe BERT, document-level embeddings LSTM CRF CoNLL03: 93.”
• Template-Based Named Entity Recognition Using BART (2021, background): “…to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al., 2016; Ma and Hovy, 2016; Luo et al., 2020) output layers to assign named entity tags (e.g. organization, person and location) or non-entity tags on each input token.”
• SpanNER: Named Entity Re-/Recognition as Span Prediction (2021, background): “NER as Different Tasks Although NER is commonly formulated as a sequence labeling task (Chiu and Nichols, 2015; Huang et al., 2015; Ma and Hovy, 2016; Lample et al., 2016; Akbik et al., 2018; Peters et al., 2018; Devlin et al., 2018; Xia et al., 2019; Akbik et al., 2019; Luo et al., 2020; Lin et al.”