Chunking with Support Vector Machines

By Taku Kudo, Yuji Matsumoto. 2001

598 citations recorded by the source corpus.

We apply Support Vector Machines (SVMs) to identify English base phrases (chunks). SVMs are known to achieve high generalization performance even with input data of high dimensional feature spaces. Furthermore, by the Kernel principle, SVMs can carry out training with smaller computational overhead independent of their dimensionality. We apply weighted voting of 8 SVMs-based systems trained with distinct chunk representations. Experimental results show that our approach achieves higher accuracy than previous approaches.

Subfields: Artificial Intelligence, Natural Language Processing, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Optimization & Training

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, methodology): “…95.23% Sha and Pereira (2003) 94.29% Kudo and Matsumoto (2001) 93.91% (b) CHUNK System F1 Ando and Zhang (2005) 89.31% Florian et al. (2003) 88.76% Kudo and Matsumoto (2001) 88.31% (c) NER System F1 Koomen et al. (2005) 77.92% Pradhan et al. (2005) 77.30% Haghighi et al. (2005) 77.04% (d)…”
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks (2016, neutral): “78 Kumar et al. (2016) 97.”
Search-based structured prediction (2009, neutral): “The second variety is typiﬁed by maximum entropy Markov models [37], though the basic idea of MEMMs has also been applied more generally to SVMs [27,28,20].”
Shallow Parsing with Conditional Random Fields (2003, methodology): “F score SVM combination 94.39% (Kudo and Matsumoto, 2001) CRF 94.38% Generalized winnow 93.89% (Zhang et al., 2002) Voted perceptron 94.09% MEMM 93.70%”
Kernel Methods and Machine Learning (2014, methodology): “The chunking approach involves decomposing the original optimization problem into subproblems of manageable size and then solving the subproblems iteratively [132].”
Spoken language understanding from unaligned data using discriminative classification models (2009, background): “Since the ATIS database is relatively large, all categories are treated as non-enumerable.1 Since many semantic trees contain multiple identical concepts, the maximum tuple length parameter is set to 3 to avoid ambiguous anchor points (see Section 2.4).”
Support Vector Learning for Semantic Argument Classification (2005, methodology): “The clustering algorithm uses a database of verb-directobject relations extracted by Lin (1998). The verbs were clustered into 64 classes using the probabilistic co-occurrence model of Hofmann (Hofmann and Puzicha, 1998).”
Efficiently Inducing Features of Conditional Random Fields (2002, unknown): “( Kudo & Matsumoto, 2001; Sha & Pereira, 2003; Zhang et al., 2002).”
A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing (2017, methodology): “As a greedy transition-based example, Kudo and Matsumoto (2001) first propose to use SVM (Cortes & Vapnik, 1995) for syntactic chunking, which obtains the highest reported result in the CoNLL 2000 shared task for chunking.”
Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data (2004, methodology): “Kudo and Matsumoto (2001) report an F1 of 94.”
Bidirectional Inference with the Easiest-First Strategy for Tagging Sequence Data (2005, methodology): “Indeed, sequential classification approaches with kernel support vector machines offer competitive performance in POS tagging and chunking (Gimenez and Marquez, 2003; Kudo and Matsumoto, 2001).”
Developing a Robust Part-of-Speech Tagger for Biomedical Text (2005, background): “The precision of POS tagging not only directly affects the performance of pattern-based approaches but also influences the accuracy of parsing which in general uses the POS tags on the words as part of the input [1,2].”
Latent Support Measure Machines for Bag-of-Words Data Classification (2014, background): “, natural language processing [2], information retrieval [3, 4] and data mining [5].”
Tuning support vector machines for biomedical named entity recognition (2002, methodology): “Several representations to encode region information are proposed and examined (Ramshaw and Marcus, 1995; Uchimoto et al., 2000; Kudo and Matsumoto, 2001).”