Applying Conditional Random Fields to Japanese Morphological Analysis

By Taku Kudo, Kaoru Yamamoto, Yuji Matsumoto. Conference on Empirical Methods in Natural Language Processing, 2004

950 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/62d8219e6be879d54b5602c90fdbeaff7a78ff74

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Shinji Watanabe
2. Graham Neubig
3. Xu Tan
4. Junichi Yamagishi
5. Yoshua Bengio
6. Takaaki Hori
7. Dong Yu
8. Jinyu Li

Citations in the corpus

• An Introduction to Conditional Random Fields for Relational Learning (2007, background): “…2005], segmenting addresses in Web pages [Culotta et al., 2004], finding semantic roles in text [Roth and Yih, 2005], identifying the sources of opinions [Choi et al., 2005], Chinese word segmentation [Peng et al., 2004], Japanese morphological analysis [Kudo et al., 2004], and many others.”
• Hybrid CTC/Attention Architecture for End-to-End Speech Recognition (2017, methodology): “Finally, some languages do not explicitly have a word boundary and need tokenization modules [3], [4].”
• Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation (T) (2015, methodology): “We use following tools in this study: MeCab to tokenize Japanese sentences [29], pialign to train word alignments [22], KenLM to train Kneser-Ney smoothed language model [30], Moses to train and generate target sentences using the PBMT model [31], and Travatar to train and generate target sentences ”
• Machine Translation Detection from Monolingual Web-Text (2013, methodology): “To obtain the POS information, we use Mecab (Kudo et al., 2004) for Japanese and a POS tagger developed by Toutanova et al. (2003) for English.”
• Joint CTC/attention decoding for end-to-end speech recognition (2017, methodology): “In addition to the pronunciation dictionary issue, some languages, which do not explicitly have a word boundary, need language-specific tokenization modules (Kudo et al., 2004; Bird, 2006) for language modeling.”
• Adapting and controlling DNN-based speech synthesis using input codes (2017, methodology): “For the input linguistic features, Open JTalk(4) was used to perform standard analysis of Japanese text for synthesis, including grapheme-to-phoneme conversion, part-of-speech tagging, and morphological analysis with the MeCab parser (based on [28]).”
• Towards Burmese (Myanmar) Morphological Analysis (2019, methodology): “However, conjugated forms of numerous suffixes in Japanese should also be recovered in deeper analysis because of the agglutinativeness and the syllabic writing system [24].”
• Generating More Specific Questions for Acquiring Attributes of Unknown Concepts from Users (2013, methodology): “The Japanese morphological analyzer Mecab (Kudo et al., 2004) with the IPADIC dictionary is used to segment restaurant names into word sequences.”
• DCU-Lingo24 Participation in WMT 2014 Hindi-English Translation task (2014, methodology): “Like Bengali, Hindi is a morphologically very rich and highly inflected Indian language.”
• Context-Based Morphological Disambiguation with Random Fields (2005, methodology): “The lattice-conditional estimation approach was first used by Kudo et al. (2004) for Japanese segmentation and hierarchical POS-tagging and by Smith and Smith (2004) for Korean morphological disambiguation.”
• An Introduction to Conditional Random Fields (2010, background): “Since then, linear-chain CRFs have been applied to many problems in natural language processing, including named-entity recognition [72], feature induction for NER [71], shallow parsing [108, 120], identifying protein names in biology abstracts [107], segmenting addresses in Web pages [26], informat”
• A review of natural language processing techniques for opinion mining systems (2017, methodology): “Conditional Random Fields (CRFs) [43] have been applied to this problem and outperformed hidden Markov models and maximum-entropy Markov models [44, 45, 46].”
• A hybrid approach to Vietnamese word segmentation (2016, unknown): “Other studies using CRFs [10], SVMs [11], Hidden Markov Model (HMM) [12], [13], n-gram model [14], Maximum Entropy (MaxEnt) [15], [16] and probabilistic ensemble learning [17] also produces high accuracy for Vietnamese and other East Asian languages.”
• Representing General Relational Knowledge in ConceptNet 5 (2012, methodology): “In Japanese, we use the commonly-used MeCab algorithm for splitting words and reducing the words to a dictionary form (Kudo et al., 2004), and in many European languages we use the Snowball stemmer for that language (Porter, 2001) to remove stop words and reduce inflected words to a common stem.”
• Pointwise Prediction for Robust, Adaptable Japanese Morphological Analysis (2011, methodology): “The CRF-based method presented by Kudo et al. (2004) is generally accepted as the state-of-the-art in this paradigm.”