A Gold Standard Dependency Corpus for English

By Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel R. Bowman, Miriam Connor, John Bauer, Christopher D. Manning. International Conference on Language Resources and Evaluation, 2014

335 citations recorded by the source corpus.

We present a gold standard annotation of syntactic dependencies in the English Web Treebank corpus using the Stanford Dependencies formalism. This resource addresses the lack of a gold standard dependency treebank for English, as well as the limited availability of gold standard syntactic annotations for English informal text genres. We also present experiments on the use of this resource, both for training dependency parsers and for evaluating the quality of different versions of the Stanford Parser, which includes a converter tool to produce dependency annotation from constituency trees. We show that training a dependency parser on a mix of newswire and web data leads to better performance on that type of data without hurting performance on newswire text, and therefore gold standard annotations for non-canonical text can be a valuable resource for parsing. Furthermore, the systematic annotation effort has informed both the SD formalism and its implementation in the Stanford Parser’s dependency converter. In response to the challenges encountered by annotators in the EWT corpus, the formalism has been revised and extended, and the converter has been improved.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/82cf69e48ede65b9d1f419da786c0349342d449d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Evaluating NLP Models via Contrast Sets (2020, methodology): “We use a subset of the English UD treebanks: GUM (Zeldes, 2017), the English portion of LinES (Ahrenberg, 2007), the English portion of ParTUT (Sanguinetti and Bosco, 2015), and the dependency-annotated English Web Treebank (Silveira et al., 2014).”
BERT Rediscovers the Classical NLP Pipeline (2019, methodology): “Dependencies is the English Web Treebank (Silveira et al., 2014), SPR is the SPR1 dataset of (Teichert et al.”
Correlating Neural and Symbolic Representations of Language (2019, methodology): “We use the 2,002 sentences corresponding to the development section of the EWT Universal Depen-dencies (Silveira et al., 2014), plus 200 sentences from the training section as reference sentences when ﬁtting RSA REGRESS .”
Large-scale evidence of dependency length minimization in 37 languages (2015, background): “Arabic Semitic HamleDT Prague Arabic Dependency Treebank (43) Bulgarian IE/Slavic HamleDT BulTreeBank (44) Bengali IE/Indo-Aryan HamleDT Hyderabad Dependency Treebank (45) Catalan IE/Romance HamleDT AnCora-CA (46) Chinese Sino-Tibetan — Chinese Dependency Treebank[40] Croatian IE/Slavic HamleDT+ Cro”
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020, background): “2 (Silveira et al., 2014) for syntactic dependencies, and the English portion of Ontonotes 5.”
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020, neutral): “We use a subset of the English UD tree-banks: GUM (Zeldes, 2017), the English portion of LinES (Ahrenberg, 2007), the English portion of ParTUT (Sanguinetti and Bosco, 2015), and the dependency-annotated English Web Treebank (Sil-veira et al., 2014).”
Towards a Unified Natural Language Inference Framework to Evaluate Sentence Representations (2018, background): “…et al., 2018b).7 Context sentences are simple sentences from the three 6 https://www.decomp.net 7The last dataset provides a larger coverage of White et al. (2016)’s event factuality annotations on top of the English version of the Universal Dependencies treebank (Silveira et al., 2014). datasets.”
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation (2020, background): “The dataset includes three UD English treebanks: UD English-ParTUT, a conversion of a multilingual parallel treebank consisting of a variety of text genres, including talks, legal texts and Wikipedia articles (Sanguinetti and Bosco, 2015); the Universal Dependencies version annotation from the GUM c”
Linguistic Knowledge and Transferability of Contextual Representations (2019, methodology): “We experiment with two standard datasets: the Penn Treebank (PTB; Marcus et al., 1993) and the Universal Dependencies English Web Treebank (UD-EWT; Silveira et al., 2014).”
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020, methodology): “We use the English Web Tree-bank portion of Universal Dependencies 2.2 release (Silveira et al., 2014) for this task.”
The GUM corpus: creating multilayer resources in the classroom (2016, background): “…the criticalness of creating manually annotated gold standards for Web genres and new social media for domain adaptation of automatic parsing; see Silveira et al. (2014). are initially introduced in the text, how they are referred back to, and what licenses and allows the resolution of their…”
BERTweet: A pre-trained language model for English Tweets (2020, methodology): ““+c”: additional use of the POS annotated data from the UD English-EWT training set (Silveira et al., 2014). have the same architecture conﬁguration as our BERTweet.”
Exploring and Predicting Transferability across NLP Tasks (2020, background): “, 2011) 29K NER (Tjong Kim Sang and De Meulder, 2003) 14K POS-EWT (Silveira et al., 2014) 13K Conj (Ficler and Goldberg, 2016) 13K Chunk (Tjong Kim Sang and Buchholz, 2000) 9K”
Deep RNNs Encode Soft Hierarchical Syntax (2018, methodology): “1 Dependency Parsing We train a four-layer version of the Stanford dependency parser (Dozat and Manning, 2017) on the Universal Dependencies English Web Treebank (Silveira et al., 2014).”
What do you learn from context? Probing for sentence structure in contextualized word representations (2019, methodology): “Thus, for dependencies, we use the English Web Treebank portion of the Universal Depen-dencies 2.2 release (Silveira et al., 2014).”
Competency Problems: On Finding and Removing Artifacts in Language Data (2021, methodology): “We analyze two datasets with the hypothesis test from §3.2: SNLI (Bowman et al., 2015) and the Universal Dependencies English Web Tree-bank (Silveira et al., 2014).”