The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies

By Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluís Màrquez i Villodre, Joakim Nivre. 2008

558 citations recorded by the source corpus.

The Conference on Computational Natural Language Learning is accompanied every year by a shared task whose purpose is to promote natural language processing applications and evaluate them in a standard setting. In 2008 the shared task was dedicated to the joint parsing of syntactic and semantic dependencies. This shared task not only unifies the shared tasks of the previous four years under a unique dependency-based formalism, but also extends them significantly: this year's syntactic dependencies include more information such as named-entity boundaries; the semantic dependencies model roles of both verbal and nominal predicates. In this paper, we define the shared task and describe how the data sets were created. Furthermore, we report and analyze the results and describe the approaches of the participating systems.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Graph Convolutional Encoders for Syntax-aware Neural Machine Translation (2017, background): “In principle, GCNs are ﬂexible enough to incorporate any linguistic structure as long as they can be represented as graphs (e.g., dependency-based semantic-role labeling representations (Surdeanu et al., 2008), AMR semantic graphs (Banarescu et al., 2012) and co-reference chains).”
Semantic Role Labeling (2010, unknown): “Although the book wisely presents a well-assessed and restricted set of techniques, the prolific SRL research has developed many other interesting methods, for example, in CoNLL (Carreras and Màrquez 2005; Surdeanu et al. 2008).”
Joint Entity Recognition and Disambiguation (2015, methodology): “The CoNLL 2008 shared task (Surdeanu et al., 2008) was intended to encourage jointly optimize parsing and semantic role labeling, but the top performing systems decoupled the two tasks.”
A High-Performance Syntactic and Semantic Dependency Parser (2010, methodology): “In combination with the system’s lem-matizer and POS tagger, this parser achieves an average labeled attachment score (LAS) of 89.88 when trained and tested on the English corpus of the CoNLL 2009 shared task (Surdeanu et al., 2008).”
The First Surface Realisation Shared Task: Overview and Evaluation Results (2011, methodology): “In addition to these atomic labels, edges can be labeled with non-atomic labels, which consist of multiple atomic labels (see Surdeanu et al. (2008) for details).”
CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes (2012, methodology): “The computational learning community is also witnessing a shift towards joint inference based evaluations, with the two previous CoNLL tasks (Surdeanu et al., 2008; Hajič et al., 2009) devoted to joint learning of syntactic and semantic dependencies.”
Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies (2009, background): “Our thanks give to the following corpus providers, (Taulé et al., 2008; Palmer and Xue, 2009; Hajič et al., 2006; Surdeanu et al., 2008; Burchardt et al., 2006) and (Kawahara et al.”
Stack-Pointer Networks for Dependency Parsing (2018, methodology): “For the CoNLL Treebanks, we use the English treebank from CoNLL-2008 shared task (Surdeanu et al., 2008) and all 13 treebanks from CoNLL-2006 shared task (Buchholz and Marsi, 2006).”
Discriminative Syntax-Based Word Ordering for Text Generation (2015, unknown): “The data are based on the CoNLL 2008 shared task data (Surdeanu et al. 2008), which consist of selected sections of the Penn WSJ Treebank, converted to syntactic dependencies via the LTH tool (Johansson and Nugues 2007).”
The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages (2009, methodology): “The complete merging process and the conversion from the constituent representation to dependencies is detailed in (Surdeanu et al., 2008).”
A Survey of the Usages of Deep Learning for Natural Language Processing (2020, result): “While this model did produce results within 2% of the state of the art (on the Wall Street Journal portion of the CoNLL 2008 Shared Task data set [103]), by the time it reached the end of a sentence, it seemed to have difficulty in remembering phrases from early in the sentence.”
Semantic Role Labeling with Neural Network Factors (2015, background): “The CoNLL 2008-2009 shared tasks introduced a variant where semantic dependencies are annotated rather than phrasal arguments (Surdeanu et al., 2008; Hajič et al., 2009).”
Multilingual Semantic Role Labeling (2009, methodology): “It obtained the best F1 score on the Chinese and German data and the second best on English.”
High-Order Low-Rank Tensors for Semantic Role Labeling (2015, methodology): “We evaluate our tensor-based approach for SRL on the CoNLL–2009 shared task benchmark datasets of ﬁve languages: English, German, Chinese, Catalan and Spanish (Surdeanu et al., 2008).”
Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs (2016, methodology): “4 Experimental Setup Our model is evaluated on the CoNLL shared tasks on joint syntactic and semantic dependency parsing in 2008 (Surdeanu et al., 2008) and 2009 (Hajič et al.”
Turning on the Turbo: Fast Third-Order Non-Projective Turbo Parsers (2013, methodology): “3), we used 14 datasets, most of which are non-projective, from the CoNLL 2006 and 2008 shared tasks (Buchholz and Marsi, 2006; Surdeanu et al., 2008).”
Universal Conceptual Cognitive Annotation (UCCA) (2013, methodology): “In fact, the annotations of (a) and (c) are identical under the most widely-used schemes for English, the Penn Treebank (PTB) (Marcus et al., 1993) and CoNLL-style dependencies (Surdeanu et al., 2008) (see Figure 1).”