CoNLL-X Shared Task on Multilingual Dependency Parsing

By Sabine Buchholz, Erwin Marsi. Conference on Computational Natural Language Learning, 2006

1,111 citations recorded by the source corpus.

Each year the Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their systems on exactly the same data sets, in order to better compare systems. The tenth CoNLL (CoNLL-X) saw a shared task on Multilingual Dependency Parsing. In this paper, we describe how treebanks for 13 languages were converted into the same dependency format and how parsing performance was measured. We also give an overview of the parsing approaches that participants took and the results that they achieved. Finally, we try to draw general conclusions about multi-lingual parsing: What makes a particular language, treebank or annotation scheme easier or harder to parse and which phenomena are challenging for any dependency parser?

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/1deed1a4a03e07aee3b8b8e4716f35033c715a57

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Neural Probabilistic Model for Non-projective MST Parsing (2017, background): “, 2008) and all 13 treebanks from CoNLL-2006 shared task (Buchholz and Marsi, 2006).”
Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs (2015, methodology): “We also experimented with the Turkish dependency treebank4 (Oflazer et al., 2003) of the CoNLL-X Shared Task (Buchholz and Marsi, 2006).”
A Latent Variable Model for Generative Dependency Parsing (2007, methodology): “The CoNLL-2006 share d t sk [7] made a wide selection of standardized treebanks for different languag es available for the research community and allowed for easy comparison between various s tatistical methods on a standardized benchmark.”
Stack-Pointer Networks for Dependency Parsing (2018, methodology): “There are two dominant approaches to dependency parsing (Buchholz and Marsi, 2006; Nivre et al., 2007): local and greedy transitionbased algorithms (Yamada and Matsumoto, 2003; Nivre and Scholz, 2004; Zhang and Nivre, 2011; Chen and Manning, 2014), and the globally optimized graph-based algorithms (”
An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing (2010, background): “An important part of this research effort are the CoNLL 2006 and 2007 shared tasks (Buchholz and Marsi, 2006; Nivre et al., 2007), which allowed for a comparison of many algorithms and approaches for this task on many languages.”
Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings (2016, methodology): “The training data for SVM comes from a combination of the Universal Dependencies Treebanks, CoNLL-X, and CoNLL-07 datasets (Buchholz and Marsi, 2006; Nilsson et al., 2007), excluding all sentences in the target language.”
brat: a Web-based Tool for NLP-Assisted Text Annotation (2012, background): “…of tasks, involving texts in seven different languages and including examples from corpora such as those introduced for the CoNLL shared tasks on language-independent named entity recognition (Tjong Kim Sang and De Meulder, 2003) and multilingual dependency parsing (Buchholz and Marsi, 2006).”
The Effect of Syntactic Representation on Semantic Role Labeling (2008, background): “Dependency syntax has only received little attention for the SRL task, despite a surge of interest in dependency parsing during the last few years (Buchholz and Marsi, 2006).”
Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning (2016, methodology): “We further study the effect of monolingual heterogeneous treebanks as source using UDT and the C O NLL-X shared task dataset (Buchholz and Marsi, 2006).”
Using Universal Linguistic Knowledge to Guide Grammar Induction (2010, methodology): “dencies with the Collins head finding rules (Collins, 1999); for the other languages we use data from the 2006 CoNLL-X Shared Task (Buchholz and Marsi, 2006).”
Introducing the SPMRL 2014 Shared Task on Parsing Morphologically-rich Languages (2014, result): “Compared to previous multilingual shared tasks (Buchholz and Marsi, 2006; Nivre et al., 2007), the SPMRL shared task targeted parsing in realistic evaluation scenarios, in which the analysis of morphologically ambiguous input tokens is not known in advance.”
CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (2017, result): “While the two tasks (Buchholz and Marsi, 2006; Nivre et al., 2007) were extremely important in setting the scene for the following years, there were also limitations that complicated application of their results: (1) gold-standard to-”
Exponentiated Gradient Algorithms for Conditional Random Fields and Max-Margin Markov Networks (2008, positive): “We report results on the Spanish dataset which is part of the CoNLL-X Shared Task on multilingual dependency parsing (Buchholz and Marsi, 2006).”
Improving Arabic Dependency Parsing with Lexical and Inflectional Morphological Features (2010, background): “Besides the work we describe in §3, Nivre (2008) reports experiments on Arabic parsing using his MaltParser (Nivre et al., 2007), trained on the PADT.”
Structured Training for Neural Network Transition-Based Parsing (2015, background): “Lately, dependency parsing has emerged as a popular approach to this problem due to the availability of dependency treebanks in many languages (Buchholz and Marsi, 2006; Nivre et al., 2007; McDonald et al., 2013) and the efficiency of dependency parsers.”