MBT: A Memory-Based Part of Speech Tagger-Generator

By Walter Daelemans, Jakub Zavrel, Peter Berck, Steven Gillis. 1996

We introduce a memory-based approach to part of speech tagging. Memory-based learning is a form of supervised learning based on similarity-based reasoning. The part of speech tag of a word in a particular context is extrapolated from the most similar cases held in memory. Supervised learning approaches are useful when a tagged corpus is available as an example of the desired output of the tagger. Based on such a corpus, the tagger-generator automatically builds a tagger which is able to tag new text the same way, diminishing development time for the construction of a tagger considerably. Memory-based tagging shares this advantage with other statistical or machine learning approaches. Additional advantages specific to a memory-based approach include (i) the relatively small tagged corpus size sufficient for training, (ii) incremental learning, (iii) explanation capabilities, (iv) flexible integration of information in case representations, (v) its non-parametric nature, (vi) reasonably good results on unknown words without morphological analysis, and (vii) fast learning and tagging. In this paper we show that a large-scale application of the memory-based approach is feasible: we obtain a tagging accuracy that is on a par with that of known statistical approaches, and with attractive space and time complexity properties when using IGTree, a tree-based formalism for indexing and searching huge case bases. The use of IGTree has as additional advantage that optimal context size for disambiguation is dynamically computed. 1 I n t r o d u c t i o n Part of Speech (POS) tagging is a process in which syntactic categories are assigned to words. It can be seen as a mapping from sentences to strings of tags. Automatic tagging is useful for a number of applications: as a preprocessing stage to parsing, in information retrieval, in text to speech systems, in corpus linguistics, etc. The two factors determining the syntactic category of a word are its lexical probability (e.g. without context, man is more probably a noun than a verb), and its contextual probability (e.g. after a pronoun, man is more probably a verb than a noun, as in they man the boats). Several approaches have been proposed to construct automatic taggers. Most work on statistical methods has used n-gram models or Hidden Markov Model-based taggers (e.g. Church, 1988; DeRose, 1988; Cutting et al. 1992; Merialdo, 1994, etc.). In

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Generalization through Memorization: Nearest Neighbor Language Models (2019, methodology): “Nearest neighbor models have been applied to a number of NLP problems in the past, such as part of speech tagging (Daelemans et al., 1996) and morphological analysis (Bosch et al., 2007), but the use of learned representations makes the similarity function much more effective in the case of neural…”
Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition (2002, neutral): “The data consists of words, entity tags and part-of-speech tags which have been derived by a Dutch part-of-speech tagger (Daelemans et al., 1996).”
Semi-supervised condensed nearest neighbor for part-of-speech tagging (2011, background): “Memory-based learning has been applied to a wide range of natural language processing tasks including part-of-speech tagging (Daelemans et al., 1996), dependency parsing (Nivre, 2003) and word sense disambiguation (K¨ubler and Zhekova, 2009).”
The effects of part–of–speech tagging on text–to–speech synthesis for resource–scarce languages (2010, methodology): “conducted experiments with their MBL tagger on the Penn Treebank WSJ corpus in [22].”
Part-of-Speech Tagging Using Decision Trees (1998, methodology): “…(Brill 95) automatically learn a set of transformation rules which best repair the errors commited by a most{likely{tag tagger, (Samuelsson et al. 96) acquires Constraint Grammar rules from tagged corpora and (Daelemans et al. 96) or the work presented here learn decision trees from tagged corpora.”
Sentence Simplification by Monolingual Machine Translation (2012, methodology): “We use the Memory-Based Tagger (Daelemans et al., 1996) trained on the Brown corpus to compute the part-ofspeech tags.”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, methodology): “K-NN based approach in other NLP applications: Nearest neighbor models have been used in a number of NLP applications such as parts-of-speech tagging (Daelemans et al., 1996) and morphological analysis (Bosch et al.”