Overview of the 2012 Shared Task on Parsing the Web

By Slav Petrov, Ryan T. McDonald. 2012

188 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Grammar as a Foreign Language (2014, methodology): “But it is not representative for text encountered on the web [8].”
Does String-Based Neural MT Learn Source Syntax? (2016, neutral): “(Pradhan and Xue, 2009) and the English Web Tree-bank (Petrov and McDonald, 2012).”
Globally Normalized Transition-Based Neural Networks (2016, methodology): “…experiments on a number of different datasets: (1) the English Wall Street Journal (WSJ) part of the Penn Treebank (Marcus et al., 1993) with standard POS tagging splits; (2) the English “Treebank Union” multi-domain corpus containing data from the OntoNotes corpus version 5 (Hovy et al.,…”
Learning to select data for transfer learning with Bayesian Optimization (2017, methodology): “For POS tagging and parsing, we evaluate on the coarse-grained POS data (12 universal POS) of the SANCL 2012 shared task (Petrov and McDonald, 2012).”
Structured Training for Neural Network Transition-Based Parsing (2015, methodology): “We also investigated the impact of connecting the final perceptron layer to all prior hidden layers (Table 6).”
A Gold Standard Dependency Corpus for English (2014, methodology): “This particular release was chosen because the dependencies produced by it were used as a gold standard for the evaluation other systems in the 2012 First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL) (Petrov and McDonald, 2012).”
Comparative Study of CNN and RNN for Natural Language Processing (2017, methodology): “We use the setup of (Blitzer et al., 2006; Petrov and McDonald, 2012): sections 2-21 are train, section 22 is dev and section 23 is test.”
Human-in-the-Loop Parsing (2016, background): “The size of labelled datasets has long been recognized as a bottleneck in the performance of natural language processing systems (Marcus et al., 1993; Petrov and McDonald, 2012).”
Cross-lingual syntactic variation over age and gender (2015, methodology): “Expected parse performance can be estimated from the SANCL 2012 shared task on dependency parsing of web data (Petrov and McDonald, 2012).”
Tailoring Continuous Word Representations for Dependency Parsing (2014, methodology): “We report signiﬁcant improvements over our base-line on both the Penn Treebank (PTB; Marcus et al., 1993) and the English Web treebank (Petrov and McDonald, 2012).”
Feature Noising for Log-Linear Structured Prediction (2013, methodology): “For the SANCL (Petrov and McDonald, 2012) POS tagging task, we used the same CRF framework with a much simpler set of features • word unigrams: w−1, w0, w1 • word bigram: (w−1, w0) and (w0, w1) We obtained a small but consistent improvement using the quadratic dropout regularizer in (14) over the…”
Equity Beyond Bias in Language Technologies for Education (2019, background): “Many of these problems stem from training data selection; models trained on standard written professional English, like the Penn Treebank (Marcus et al., 1993), fail to transfer to other writing styles, especially online where research suggests that NLP performance is degraded for under-represented ”
Modelling input texts: from Tree Kernels to Deep Learning (2015, result): “This is in line with recent advances in parsing “The Web” [112], where participants were asked to build a single system able to cope with different yet related domains.”
A Survey of the Usages of Deep Learning for Natural Language Processing (2020, methodology): “[99] created an LSTM with an attention mechanism in a syntactic constituency parser, which they tested on data from domains different from those of the test data (the English Web Treebank [100] and the Question Treebank [101] as opposed to the Wall Street Journal portion of the PTB [54]), showing th”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “One example is the SANCL shared task (Petrov and McDonald, 2012), where participants were asked to build a single system that can robustly parse reviews, weblogs, answers, emails, newsgroups.”
Cross-Domain Generalization of Neural Constituency Parsers (2019, methodology): “, 2005), the English Web Treebank (Petrov and McDonald, 2012)).”