Is it Harder to Parse Chinese, or the Chinese Treebank?

By Roger Levy, Christopher D. Manning. Annual Meeting of the Association for Computational Linguistics, 2003

294 citations recorded by the source corpus.

We present a detailed investigation of the challenges posed when applying parsing models developed against English corpora to Chinese. We develop a factored-model statistical parser for the Penn Chinese Treebank, showing the implications of gross statistical differences between WSJ and Chinese Tree-banks for the most general methods of parser adaptation. We then provide a detailed analysis of the major sources of statistical parse errors for this corpus, showing their causes and relative frequencies, and show that while some types of errors are due to difficult ambiguities inherent in Chinese grammar, others arise due to treebank annotation practices. We show how each type of error can be addressed with simple, targeted changes to the independence assumptions of the maximum likelihood-estimated PCFG factor of the parsing model, which raises our F1 from 80.7% to 82.6% on our development set, and achieves parse accuracy close to the best published figures for Chinese parsing.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2f6070f88ecca9a4d20167571a856d6884d6cb1a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Phrase Dependency Machine Translation with Quasi-Synchronous Tree-to-Tree Features (2014, methodology): “…from ylk to y j i: maxu:i≤u≤j,k≤τy (u)≤l |τy (u)− u| > max v:k≤v≤l,i≤τy (v)≤j |τy (v)− v| The final condition also implies that there is a lexical dependency from a word in yji to a word in ylk: ∃u, i ≤ u ≤ j, such that k ≤ τy (u) ≤ l. 7 Other ways of getting phrase dependencies are possible.”
Syntactic Processing Using the Generalized Perceptron and Beam Search (2011, methodology): “The rows in the table represent the models from Bikel and Chiang (2000), Levy and Manning (2003), Xiong et al. (2005), Bikel (2004), Chiang and Bikel (2002), the SVM model from Wang et al. (2006), the ensemble system from Wang et al. (2006), and the parser of this article, respectively.”
Self-Training PCFG Grammars with Latent Annotations Across Languages (2009, methodology): “As pointed out in (Levy and Manning, 2003), there are many linguistic differences between Chinese and English, as well as structural differences between their corresponding treebanks, and some of these make it a harder task to parse Chinese.”
The Penn Chinese TreeBank: Phrase structure annotation of a large corpus (2005, methodology): “Its efficacy for training statistical parsers has been validated by the development of several different systems (Bikel and Chiang 2000; Levy and Manning 2003; Luo 2003).”
Mandarin Prosodic Phrase Prediction based on Syntactic Trees (2016, methodology): “In which, the POS, Syntactic Structure, and Dependency Structure are obtained using Stanford Parser [20, 21].”
Automated Phrase Mining from Massive Text Corpora (2017, methodology): “Parsers of all three languages are available in Stanford NLP tools [8], [20], [28].”
On the parameter space of generative lexicalized statistical parsing models (2004, result): “Levy et al. were employing a factored model (Klein and Manning, 2002b), which combines an unlexicalized PCFG model with a dependency model involving no constituent labels. They found a significant improvement in parse accuracy with their grammar transformations, which dealt not only with the NP-NP m”
Minimax Optimal Procedures for Locally Private Estimation (2016, methodology): “The datasets were constructed as follows: all blog posts from http://weiboscope.jmsc.hku.hk/datazip/ were down-loaded (see Fu, Chan, and Chau 2013), and the Chinese text of each post is segmented using the Stanford Chinese language parser (Levy and Manning 2003).”