Exploration of Noise Strategies in Semi-supervised Named Entity Classification

By P. Narayan, Ajay Nagesh, Mihai Surdeanu. 2019

10 citations recorded by the source corpus.

Noise is inherent in real world datasets and modeling noise is critical during training as it is effective in regularization. Recently, novel semi-supervised deep learning techniques have demonstrated tremendous potential when learning with very limited labeled training data in image processing tasks. A critical aspect of these semi-supervised learning techniques is augmenting the input or the network with noise to be able to learn robust models. While modeling noise is relatively straightforward in continuous domains such as image classification, it is not immediately apparent how noise can be modeled in discrete domains such as language. Our work aims to address this gap by exploring different noise strategies for the semi-supervised named entity classification task, including statistical methods such as adding Gaussian noise to input embeddings, and linguistically-inspired ones such as dropping words and replacing words with their synonyms. We compare their performance on two benchmark datasets (OntoNotes and CoNLL) for named entity classification. Our results indicate that noise strategies that are linguistically informed perform at least as well as statistical approaches, while being simpler and requiring minimal tuning.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “Method Following related work (Wang et al., 2019; Lakshmi Narayan et al., 2019; Liu et al., 2019b), we add zero-mean Gaussian noise N (0 , σ 2 ) to each input embedding in the I → OR encoder at inference time.”
MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification (2020, background): “Other work which also investigates noise and its incorporation into semi-supervised named entity clas-siﬁcation (Lakshmi Narayan et al., 2019; Nagesh and Surdeanu, 2018).”