Max-Margin Tensor Neural Network for Chinese Word Segmentation

By Wenzhe Pei, Tao Ge, Baobao Chang. 2014

168 citations recorded by the source corpus.

Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability to alleviate the burden of manual feature engineering. In this paper, we propose a novel neural network model for Chinese word segmentation called Max-Margin Tensor Neural Network (MMTNN). By exploiting tag embeddings and tensorbased transformation, MMTNN has the ability to model complicated interactions between tags and context characters. Furthermore, a new tensor factorization approach is proposed to speed up the model and avoid overfitting. Experiments on the benchmark dataset show that our model achieves better performances than previous neural network models and that our model can achieve a competitive performance with minimal feature engineering. Despite Chinese word segmentation being a specific case, MMTNN can be easily generalized and applied to other sequence labeling tasks.

Subfields: Artificial Intelligence, Computational Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Yonatan Belinkov
5. Jie Yang
6. Mohit Bansal
7. Eduard H. Hovy
8. Yoav Goldberg

In Computational Mathematics

1. Yonatan Belinkov
2. Eduard H. Hovy
3. Jie Yang
4. Mark Dredze
5. Fei Dong
6. Jiwei Li
7. Nanyun Peng
8. Tie-Yan Liu

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “For character-based languages such as Chinese, word segmentation [394, 444, 261] is necessary to detect the word boundary from raw text, which is important to ensure the accuracy for later POS tagging, prosody prediction, and grapheme-to-phoneme conversion process.”
• A Model of Coherence Based on Distributed Sentence Representation (2014, methodology): “To minimize the objective J(Θ), we use the diagonal variant of AdaGrad (Duchi et al., 2011) with minibatches, which is widely applied in deep learning literature (e.g.,(Socher et al., 2011a; Pei et al., 2014)).”
• An Effective Neural Network Model for Graph-based Dependency Parsing (2015, background): “Similar ideas were also proposed in previous works (Socher et al., 2013; Pei et al., 2014; Chen and Manning, 2014).”
• Neural Word Segmentation with Rich Pretraining (2017, methodology): “For the former, we follow prior methods (Xue et al., 2003; Pei et al., 2014), using five-character window [c−2, c−1, c0, c1, c2] to represent incoming characters.”
• On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, methodology): “For example, these embeddings were used in areas as diverse as sentiment analysis [8, 92], information retrieval [242], metaphor recognition [244], factoid [163] and community question answering [31, 246], summarization [169], semantic parsing [35], machine translation [354], dependency parsing [21,”
• Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings (2015, methodology): “This is a standard method that has been previously explored in sequential and structured prediction problem (Collobert et al., 2011; Zheng et al., 2013; Yao et al., 2014; Pei et al., 2014).”