A Stochastic Japanese Morphological Analyzer Using a Forward-DP Backward-A* N-Best Search Algorithm
By Masaaki Nagata. 1994
We present a novel method for segmenting the input sentence into words and assigning parts of speech to the words. It consists of a statistical language model and an efficient two-pass N-best search algorithm. The algorithm does not require delimiters between words. Thus it is suitable for written Japanese. The proposed Japanese morphological analyzer achieved 95.1% recall and 94.6% precision for open text when it was trained and tested on the ATR Corpus.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Graham Neubig
- Taku Kudo
- Felix Stahlberg
- Shinsuke Mori
- Yosuke Nakata
- Orhan Firat
- Luke S. Zettlemoyer
- Mohit Bansal
Citations in the corpus
- Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018, methodology): “More specifically, we first obtain l -best segmentations according to the probability P ( x | X ) . l -best search is performed in linear time with the Forward-DP Backward-A* al-gorithm (Nagata, 1994).”
- Pointwise Prediction for Robust, Adaptable Japanese Morphological Analysis (2011, methodology): “Previous approaches have used structured predictors such as hidden Markov models (HMMs) or conditional random fields (CRFs), which consider the interactions between neighboring words and parts of speech (Nagata, 1994; Asahara and Matsumoto, 2000; Kudo et al., 2004).”