HHMM-based Chinese Lexical Analyzer ICTCLAS

By Huaping Zhang, Hongkui Yu, Deyi Xiong, Qun Liu. 2003

485 citations recorded by the source corpus.

This document presents the results from Inst. of Computing Tech., CAS in the ACL SIGHAN-sponsored First International Chinese Word Segmentation Bake-off. The authors introduce the unified HHMM-based frame of our Chinese lexical analyzer ICTCLAS and explain the operation of the six tracks. Then provide the evaluation results and give more analysis. Evaluation on ICTCLAS shows that its performance is competitive. Compared with other system, ICTCLAS has ranked top both in CTB and PK closed track. In PK open track, it ranks second position. ICTCLAS BIG5 version was transformed from GB version only in two days; however, it achieved well in two BIG5 closed tracks. Through the first bakeoff, we could learn more about the development in Chinese word segmentation and become more confident on our HHMM-based approach. At the same time, we really find our problems during the evaluation. The bakeoff is interesting and helpful.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jamie Callan
2. Chenyan Xiong
3. Zhiyuan Liu
4. Maosong Sun
5. Zhuyun Dai
6. Zhenghao Liu
7. Jimmy J. Lin
8. Luyu Gao

In Natural Language Processing

1. Jamie Callan
2. Zhiyuan Liu
3. Chenyan Xiong
4. Maosong Sun
5. Zhuyun Dai
6. Zhenghao Liu
7. Hang Li
8. Zhengdong Lu

Citations in the corpus

• End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, methodology): “Chinese text was segmented using the open source so‰ware ICTCLAS [23].”
• An CNN-LSTM Attention Approach to Understanding User Query Intent from Online Health Communities (2017, methodology): “We also use tokenizer [17] for the word segmentation and POS tagging and the model [18] trained by our NER corpus for NER tagging.”
• Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval (2018, background): “All Chinese texts are segmented by ICTCLAS (Zhang et al., 2003), after that they are treated the same as English.”
• Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search (2018, unknown): “Chinese text was segmented by ICTCLASS [31]; then Chinese words were treated like English words.”
• SUBTLEX-CH: Chinese Word and Character Frequencies Based on Film Subtitles (2010, methodology): “. These algorithms are trained on a tagged corpus (i.e., a corpus in which all the words have been identified and given their correct syntactic role) and are then applied to new materials”
• A Review of Sentiment Analysis Research in Chinese Language (2017, unknown): “Zhang Huaping and is considered as the benchmark in Chinese word segmentation [27].”
• An Information Retrieval Approach to Short Text Conversation (2014, methodology): “We use ICTCLAS (Zhang et al., 2003) to obtain the part of speech and named entity information and LIBLINEAR (Fan et al., 2008) to build the logistic regression model and predict the probability of a word being a topic word.”