A High-Performance Semi-Supervised Learning Method for Text Chunking

By Rie Kubota Ando, Tong Zhang. 2005

250 citations recorded by the source corpus.

In machine learning, whether one can build a more accurate classifier by using unlabeled data (semi-supervised learning) is an important issue. Although a number of semi-supervised methods have been proposed, their effectiveness on NLP tasks is not always clear. This paper presents a novel semi-supervised method that employs a learning paradigm which we call structural learning. The idea is to find "what good classifiers are like" by learning from thousands of automatically generated auxiliary classification problems on unlabeled data. By doing so, the common predictive structure shared by the multiple classification problems can be discovered, which can then be used to improve performance on the target problem. The method produces performance higher than the previous best results on CoNLL'00 syntactic chunking and CoNLL'03 named entity chunking (English and German).

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semi-supervised sequence tagging with bidirectional language models (2017, background): “, 2000; Mohit and Hwa, 2005), structural learning (Ando and Zhang, 2005) and maximum discriminant functions (Suzuki et al.”
Domain Adaptation with Structural Correspondence Learning (2006, methodology): “The ASO and structural correspondence results also use projection features learned using ASO and SCL. Section 7.1 presents results comparing structural correspondence learning with the supervised baseline and ASO in the case where we have no labeled data in the target domain.”
A Primer on Neural Network Models for Natural Language Processing (2015, background): “For work using neural network architecture with sparse vector encodings see (Johnson & Zhang, 2015).”
Design Challenges and Misconceptions in Named Entity Recognition (2009, background): “Recent successful semi-supervised systems (Ando and Zhang, 2005; Suzuki and Isozaki, 2008) have illustrated that unlabeled text can be used to improve the performance of NER systems.”
ExplainaBoard: An Explainable Leaderboard for NLP (2021, methodology): “, 2003), text chunking (Ando and Zhang, 2005), and Chinese word segmentation (Chen et al.”
A Survey on Transfer Learning (2010, background): “A second case can be referred to as feature-representation-transfer approach [22], [36], [37], [38], [39], [8], [40], [41], [42], [43], [44].”
Cross-Language Text Classification Using Structural Correspondence Learning (2010, methodology): “An alternative view of cross-language structural correspondence learning is provided by the framework of structural learning (Ando and Zhang, 2005a).”
Toward Mention Detection Robustness with Recurrent Neural Networks (2016, background): “…entity recognition (Bikel et al., 1997; Borthwick et al., 1997; Tjong Kim Sang and De Meulder, 2003; Florian et al., 2003; Miller et al., 2004; Ando and Zhang, 2005; Suzuki and Isozaki, 2008; Ratinov and Roth, 2009; Lin and Wu, 2009; Turian et al., 2010; Ritter et al., 2011; Passos et al.,…”
Describing objects by their attributes (2009, background): “Several researchers [2, 18, 12, 1, 22, 17] have shown that sharing features across multiple tasks or categories can lead to increased performance, especially when training data is limited.”
Open-Source Tools for Morphology, Lemmatization, POS Tagging and Named Entity Recognition (2014, unknown): “For example, the systems which published high scores on the CoNLL-2003 task include (Suzuki and Isozaki, 2008), (Ando and Zhang, 2005) and to our knowledge, the best currently known results on this dataset were published by (Ratinov and Roth, 2009).”
Word Representations: A Simple and General Method for Semi-Supervised Learning (2010, methodology): “We compare to the state-of-the-art methods of Ando and Zhang (2005), Suzuki and Isozaki (2008), and—for NER—Lin and Wu (2009). Tables 2 and 3 show that accuracy can be increased further by combining the features from different types of word representations.”
Distributional Representations for Handling Sparsity in Supervised Sequence-Labeling (2009, background): “Ando and Zhang develop a semi-supervised chunker that outperforms purely supervised approaches on the CoNLL 2000 dataset (Ando and Zhang, 2005).”
Domain Generalization by Marginal Transfer Learning (2017, unknown): “ilable for the test data, and the training data sets are used to improve the performance of a standard classier, for example by learning a metric or embedding which is appropriate for all data sets (Ando and Zhang, 2005; Rettinger et al., 2006). In our case, no test labels are available, but we hope”
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data (2005, result): “A short version of this paper, mainly reporting some empirical results, appeared in ACL (Ando and Zhang, 2005).”
Bootstrapping Without the Boot (2005, methodology): “An example is a token of drug .”
Applying Alternating Structure Optimization to Word Sense Disambiguation (2006, methodology): “Based on these observations, we study a new application of Alternating Structure Optimization (ASO) (Ando and Zhang, 2005a; Ando and Zhang, 2005b) to WSD. ASO is a recently proposed machine learning method for learning predictive structure (i.e., information useful for predictions) shared by…”