Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop

By Nizar Habash, Owen Rambow. Annual Meeting of the Association for Computational Linguistics, 2005

533 citations recorded by the source corpus.

We present an approach to using a morphological analyzer for tokenizing and morphologically tagging (including part-of-speech tagging) Arabic words in one process. We learn classifiers for individual morphological features, as well as ways of using these classifiers to choose among entries from the output of the analyzer. We obtain accuracy rates on all tasks in the high nineties.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/dd12392e21999ff795eed1f80a234f92fe045570

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Arabic Tokenization System (2007, methodology): “These solutions include morphological analysis (Beesley 2001; Buckwalter 2002), diacritization (Nelken and Shieber 2005), Information Retrieval (Larkey and Connell 2002), and POS Tagging (Diab et al 2004; Habash and Rambow 2005).”
Arabic Natural Language Processing: Challenges and Solutions (2009, methodology): “One of the early morphological analyzers and generators for Arabic dialects, MAGEAD, uses an analyzer without a lexicon by exploiting regularities among dialects through sound changes at the radical level and thus, explicit analysis of roots and patterns [Habash and Owen 2005].”
Syntactic Phrase Reordering for English-to-Arabic Statistical Machine Translation (2009, methodology): “We use the Morphological Analyzer MADA (Habash and Rambow, 2005) to decompose the Arabic source.”
Book Reviews: Introduction to Arabic Natural Language Processing by Nizar Y. Habash (2010, methodology): “1 MADA+TOKAN Mada (Morphological Analysis and Disambiguation for Arabic) is a utility that, given raw Arabic text, adds as much lexical and morphological information as possible by disambiguating, in one operation, part-of-speech tags, lexemes, diacritizations and full morphological analyses [51, 35”
An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation (2006, methodology): “Elworthy (1994), in contrast, reports an accuracy of 75.49%, 80.87% and 79.12% for unsupervised word-based HMM trained on parts of the LOB corpora, with a tagset of 134 tags.”
Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging (2017, background): “, 2016), part-of-speech tagging (Diab et al., 2004; Habash and Rambow, 2005), and information retrieval (M.”
Segmentation for English-to-Arabic Statistical Machine Translation (2008, methodology): “and Sadat (2006) use the Arabic morphological analyzer MADA (Habash and Rambow, 2005) to segment the Arabic source; they propose various segmentation schemes.”
Unsupervised Morphology Rivals Supervised Morphology for Arabic MT (2012, methodology): “An additional advantage of Arabic for study is the availability of high-quality supervised seg-menters for MSA, such as MADA (Habash and Rambow, 2005), for performance comparison.”
Rich Morphology Generation Using Statistical Machine Translation (2012, methodology): “For example, Habash and Rambow (2005) report 98.2% and 98.8% for G EN and N UM , respectively, compared to our 84.2% and 87.3%.”
MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic (2014, methodology): “In this paper, we focus on two systems that are commonly used by researchers in Arabic NLP: MADA (Habash and Rambow, 2005; Roth et al., 2008; Habash et al., 2009; Habash et al., 2013) and AMIRA (Diab et al., 2007).”
Improving Arabic Dependency Parsing with Lexical and Inflectional Morphological Features (2010, methodology): “Therefore, we repeated the experiments above with POS tags predicted by the Morphological Analysis and Disambiguation for Arabic (MADA) toolkit (Habash and Rambow, 2005).”
Identifying Broken Plurals, Irregular Gender, and Rationality in Arabic Text (2012, methodology): “…tools for Arabic NLP: the Buckwalter Arabic Morphological Analyzer (BAMA) (Buckwalter, 2004) which is used in the Penn Arabic Tree Bank (PATB) (Maamouri et al., 2004), and the various POS tagging and morphological disambiguation tools trained using them (Diab et al., 2004; Habash and Rambow, 2005).”