An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation
By Meni Adler, Michael Elhadad. 2006
80 citations recorded by the source corpus.
Morphological disambiguation is the process of assigning one set of morphological features to each individual word in a text. When the word is ambiguous (there are several possible analyses for the word), a disambiguation procedure based on the word context must be applied. This paper deals with morphological disambiguation of the Hebrew language, which combines morphemes into a word in both agglutinative and fusional ways. We present an unsupervised stochastic model ‐ the only resource we use is a morphological analyzer ‐ which deals with the data sparseness problem caused by the axational morphology of the Hebrew language. We present a text encoding method for languages with axational morphology in which the knowledge of word formation rules (which are quite restricted in Hebrew) helps in the disambiguation. We adapt HMM algorithms for learning and searching this text representation, in such a way that segmentation and tagging can be learned in parallel in one step. Results on a large scale evaluation indicate that this learning improves disambiguation for complex tag sets. Our method is applicable to other languages with ax morphology.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoav Goldberg
- Omer Levy
- Yejin Choi
- Noah A. Smith
- Luke S. Zettlemoyer
- Mohit Bansal
- Regina Barzilay
- Jianfeng Gao
Citations in the corpus
- Word Segmentation, Unknown-word Resolution, and Morphological Agreement in a Hebrew Parsing System (2013, methodology): “In sequential tagging models such as Smith, Smith, and Tromble (2005), Adler and Elhadad (2006), and Bar-Haim, Sima’an, and Winter (2008) weights are assigned according to a tagging model based on linear context.”
- Unsupervised Multilingual Learning for Morphological Segmentation (2008, background): “Unsupervised Morphological Segmentation Unsupervised morphology is an active area of research (Schone and Jurafsky, 2000; Goldsmith, 2001; Adler and Elhadad, 2006; Creutz and Lagus, 2007; Dasgupta and Ng, 2007).”
- Neural Machine Translation without Embeddings (2020, methodology): “Byte vocabularies obviate the need to preprocess the text with hand-crafted languagespecific tokenizers (Koehn et al., 2007; Adler and Elhadad, 2006) and subword induction algorithms, such as BPE (Sennrich et al.”