GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling

By Yanjun Liu, Fandong Meng, Jinchao Zhang, Jinan Xu, Yufeng Chen, Jie Zhou. 2019

97 citations recorded by the source corpus.

Current state-of-the-art systems for sequence labeling are typically based on the family of Recurrent Neural Networks (RNNs). However, the shallow connections between consecutive hidden states of RNNs and insufficient modeling of global information restrict the potential performance of those models. In this paper, we try to address these issues, and thus propose a Global Context enhanced Deep Transition architecture for sequence labeling named GCDT. We deepen the state transition path at each position in a sentence, and further assign every token with a global representation learned from the entire sentence. Experiments on two standard sequence labeling tasks show that, given only training data and the ubiquitous word embeddings (Glove), our GCDT achieves 91.96 F1 on the CoNLL03 NER task and 95.43 F1 on the CoNLL2000 Chunking task, which outperforms the best reported results under the same settings. Furthermore, by leveraging BERT as an additional resource, we establish new state-of-the-art results with 93.47 F1 on NER and 97.30 F1 on Chunking.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Chunyuan Li
3. Weizhu Chen
4. Jiawei Han
5. Yue Zhang
6. Baolin Peng
7. Ricardo Henao
8. Jiaxin Huang

Citations in the corpus

• Few-Shot Class-Incremental Learning for Named Entity Recognition (2022, background): “However, many current state-of-the-art NER models are built with a CRF module (Liu et al., 2019; Chen et al., 2020; Wang et al., 2021).”
• Few-Shot Named Entity Recognition: A Comprehensive Study (2020, methodology): “Generic representations via self-supervised pre-trained language models (Devlin et al., 2019; Liu et al., 2019c) have benefited a wide range of NLP applications.”
• A Survey on Deep Learning for Named Entity Recognition (2018, background): “3% [135] CNN GloVe BERT, global embeddings GRU GRU CoNLL03: 93.”
• Template-Based Named Entity Recognition Using BART (2021, methodology): “Some methods (Chiu and Nichols, 2016; Strubell et al., 2017) treat NER as a local classification problem at each input token, while other methods use CRF (Ma and Hovy, 2016) or a sequence-to-sequence framework (Zhang et al., 2018; Liu et al., 2019).”