The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages
By Jan Hajic, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, M. A. Martí, Lluís Màrquez i Villodre, Adam Meyers, Joakim Nivre, Sebastian Padó, J. Stepánek and 4 more. 2009
625 citations recorded by the source corpus.
For the 11th straight year, the Conference on Computational Natural Language Learning has been accompanied by a shared task whose purpose is to promote natural language processing applications and evaluate them in a standard setting. In 2009, the shared task was dedicated to the joint parsing of syntactic and semantic dependencies in multiple languages. This shared task combines the shared tasks of the previous five years under a unique dependency-based formalism similar to the 2008 task. In this paper, we define the shared task, describe how the data sets were created and show their quantitative properties, report the results and summarize the approaches of the participating systems.
Subfields: Artificial Intelligence, Natural Language Processing, Computational Linguistics & Semantics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Mohit Bansal
- Noah A. Smith
- Graham Neubig
- Percy Liang
- Jianfeng Gao
- Christopher D. Manning
- Yoav Goldberg
In Natural Language Processing
- Luke S. Zettlemoyer
- Graham Neubig
- Noah A. Smith
- Mohit Bansal
- Mirella Lapata
- Christopher D. Manning
- Percy Liang
- Yoav Goldberg
In Computational Linguistics & Semantics
- Aliaksei Severyn
- Mirella Lapata
- Sebastian Padó
- Wanxiang Che
- Ting-Wei Liu
- Hao Peng
- Noah A. Smith
- Roberto Navigli
Citations in the corpus
- Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “We rely on the PropBank semantic formalism (Palmer et al., 2005), as it provides well-established representations of meanings that are generalizable across di ff erent predicates and languages (Hajiˇc et al., 2009).”
- Globally Normalized Transition-Based Neural Networks (2016, methodology): “…“Treebank Union” multi-domain corpus containing data from the OntoNotes corpus version 5 (Hovy et al., 2006), the English Web Treebank (Petrov and McDonald, 2012), and the updated and corrected Question Treebank (Judge et al., 2006) with identical setup to Weiss et al. (2015); and (3) the…”
- Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language (2015, methodology): “In the newswire domain, we sampled sentences from the English training data of CoNLL-2009 shared task (Hajič et al., 2009), excluding questions and sentences with fewer than 10 words.”
- Polyglot: Distributed Word Representations for Multilingual NLP (2013, background): “Despite recent momentum towards developing multilingual tools (Nivre et al., 2007; Hajič et al., 2009; Pradhan et al., 2012), most of NLP research still focuses on rich resource languages.”
- Efficient Higher-Order CRFs for Morphological Tagging (2013, methodology): “For Czech and Spanish, we use the CoNLL 2009 data sets (Hajič et al., 2009); for German, the TIGER treebank (Brants et al., 2002) with the split from Fraser et al. (2013); for Hungarian, the Szeged treebank (Csendes et al., 2005) with the split from Farkas et al. (2012).”
- Frame-Semantic Parsing (2014, methodology): “Experiments are performed on the CoNLL 2008 shared task data set (Surdeanu et al. 2008), which follows the PropBank conventions and only labels verbal and nominal predicates—in contrast to our work, which includes most lexicosyntactic categories. A similar approach is presented by Weston, Ratle, and”
- High-Order Low-Rank Tensors for Semantic Role Labeling (2015, methodology): “Our setup follows the CoNLL–2009 shared task (Hajiˇc et al., 2009).”
- Efficient Inference and Structured Learning for Semantic Role Labeling (2015, background): “Since then, there has been work on SRL for nominal predicates (Meyers et al., 2004; Gerber and Chai, 2010) and variants that investigated the prediction of semantic dependencies rather than phrasal arguments (Surdeanu et al., 2008; Hajiˇc et al., 2009).”
- Design and realization of a modular architecture for textual entailment (2013, background): “The columnbased CoNLL 2009 file format (Hajič et al., 2009) offers a lightweight, general representation, but lacks in extensibility: new layers must be defined ad hoc, variable annotation (such as semantic roles) leads to variable numbers of columns, and there is no declarative support for meta dat”
- Broad Coverage Multilingual Deep Sentence Generation with a Stochastic Multi-Level Realizer (2010, methodology): “In this paper, we present a Support Vector Machine (SVM)-based multilingual dependencyoriented stochastic deep sentence realizer that uses multilingual corpora of the CoNLL ’09 shared task (Hajič, 2009) for training.”
- Neural Semantic Role Labeling with Dependency Path Embeddings (2016, methodology): “We train our models on data from the CoNLL-2009 shared task, relying on the same features as one of the participating systems (Bj¨orkelund et al., 2009), and evaluate with the official scorer.”
- Semantic Role Labeling with Neural Network Factors (2015, background): “The CoNLL 2008-2009 shared tasks introduced a variant where semantic dependencies are annotated rather than phrasal arguments (Sur-deanu et al., 2008; Hajiˇc et al., 2009).”
- Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies (2009, methodology): “This paper describes the system of National Institute of Information and Communications Technology (NICT) and City University of Hong Kong (CityU) for the joint learning task of CoNLL-2009 shared task (Hajiˇc et al., 2009) 1 .”
- Multilingual Semantic Role Labeling (2009, background): “While input parse trees in Chinese and German had a labeled syntactic accuracy of 78.46 (Hajič et al., 2009), we could reach a labeled semantic F1 of 78.60 and 79.71 in these languages.”
- Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking (2020, methodology): “SRL We used the English CoNLL-2009 shared task dataset (Hajiˇc et al., 2009).”
- TectoMT: Modular NLP Framework (2010, background): “TectoMT includes converters for various formats and corpora (e.g. Penn Treebank [25], CoNLL [12], EMILLE [26], PADT [27]).”