Deep Learning for Chinese Word Segmentation and POS Tagging

By Xiaoqing Zheng, Hanyang Chen, Tianyu Xu. 2013

329 citations recorded by the source corpus.

This study explores the feasibility of performing Chinese word segmentation (CWS) and POS tagging by deep learning. We try to avoid task-specific feature engineering, and use deep layers of neural networks to discover relevant features to the tasks. We leverage large-scale unlabeled data to improve internal representation of Chinese characters, and use these improved representations to enhance supervised word segmentation and POS tagging models. Our networks achieved close to state-of-theart performance with minimal computational cost. We also describe a perceptron-style algorithm for training the neural networks, as an alternative to maximum-likelihood method, to speed up the training process and make the learning algorithm easier to be implemented.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Erik Cambria
2. Furu Wei
3. Tie-Yan Liu
4. Soujanya Poria
5. Tao Qin
6. Xu Tan
7. Devamanyu Hazarika
8. Percy Liang

In Computer Vision and Pattern Recognition

1. Soujanya Poria
2. Tao Qin
3. Tie-Yan Liu
4. Erik Cambria
5. Xu Tan
6. Furu Wei
7. Devamanyu Hazarika
8. M. Zhou

Citations in the corpus

• Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “Thus, works employing deep learning applications on such languages tend to prefer character embeddings over word vectors [34].”
• A Survey on Neural Speech Synthesis (2021, methodology): “For character-based languages such as Chinese, word segmentation [394, 444, 261] is necessary to detect the word boundary from raw text, which is important to ensure the accuracy for later POS tagging, prosody prediction, and grapheme-to-phoneme conversion process.”
• Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification (2014, background): “It is meaningful for some tasks such as pos-tagging (Zheng et al., 2013) as the two words have similar usages and grammatical roles, but it becomes a disaster for sentiment analysis as they have the opposite sentiment polarity.”
• Automatic prosody prediction for Chinese speech synthesis using BLSTM-RNN and embedding features (2015, methodology): “This architecture was subsequently used for CWS and POS tagging [12].”
• Learning Character-level Representations for Part-of-Speech Tagging (2014, background): “Recent work has showed that large improvements in terms of model accuracy can be obtained by performing unsupervised pre-training of word embeddings (Collobert et al., 2011; Luong et al., 2013; Zheng et al., 2013; Socher et al., 2013).”
• A review of natural language processing techniques for opinion mining systems (2017, background): “As the similarity and relevance among word segmentation, POS tagging, and parsing, some approaches are proposed to deal with these tasks simultaneously [50, 51, 52].”
• Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings (2015, methodology): “com/fxsjy/jieba This is a standard method that has been previously explored in sequential and structured prediction problem (Collobert et al., 2011; Zheng et al., 2013; Yao et al., 2014; Pei et al., 2014).”
• Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts (2014, methodology): “The test set was manually annotated by Go et al. (2009) (Collobert et al., 2011; Luong et al., 2013; Zheng et al., 2013; Socher et al., 2013a).”
• Neural Word Segmentation with Rich Pretraining (2017, background): “角 (corner)” (Zheng et al., 2013), which is infeasible by using sparse one-hot character features.”
• Deep Convolution Neural Networks for Twitter Sentiment Analysis (2018, background): “Recent studies have shown that the use of pre-trained word embeddings can substantially improve the performance of the model [22], [26]–[28].”
• Max-Margin Tensor Neural Network for Chinese Word Segmentation (2014, methodology): “Zheng et al. (2013) followed the model proposed by Collobert et al. (2008).”
• SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup (2020, background): “Many NLP tasks can be formulated as sequence labeling problems, such as part-of-speech (POS) tagging (Zheng et al., 2013), named entity recognition (NER) (Lample et al., 2016), and event extraction (Yang et al., 2019).”
• Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “Thus, works employing deep learning applications on such languages tend to prefer character embeddings over word vectors [34].”