The CoNLL-2014 Shared Task on Grammatical Error Correction
By Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, Christopher Bryant. 2014
580 citations recorded by the source corpus.
The CoNLL-2014 shared task was devoted to grammatical error correction of all error types. In this paper, we give the task definition, present the data sets, and describe the evaluation metric and scorer used in the shared task. We also give an overview of the various approaches adopted by the participating teams, and present the evaluation results. Compared to the CoNLL2013 shared task, we have introduced the following changes in CoNLL-2014: (1) A participating system is expected to detect and correct grammatical errors of all types, instead of just the five error types in CoNLL-2013; (2) The evaluation metric was changed from F1 to F0.5, to emphasize precision over recall; and (3) We have two human annotators who independently annotated the test essays, compared to just one human annotator in CoNLL-2013.
Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Furu Wei
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
- Felix Stahlberg
- Yejin Choi
- Graham Neubig
In Natural Language Processing
In Information Extraction
Citations in the corpus
- Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021, methodology): “For facilitating fair comparison in the efficiency evaluation, we follow the previous studies (Omelianchuk et al., 2020; Chen et al., 2020) which conduct GEC efficiency evaluation to use CoNLL-2014 (Ng et al., 2014) dataset that contains 1,312 sentences as our main test set, and evaluate the speedup a”
- Identifying Well-formed Natural Language Questions (2018, background): “Grammatical error correction (GEC) is the task of correcting the grammatical errors (if any) in a piece of text (Ng et al., 2014).”
- Semi-supervised Multitask Learning for Sequence Labeling (2017, methodology): “In addition, we evaluate on the CoNLL 2014 shared task dataset (Ng et al., 2014), which has been converted to an error detection task.”
- The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction (2019, methodology): “Many of the topranked systems in the CoNLL-2013 and 2014 GEC shared tasks (Ng et al., 2013, 2014), were either based on language models or had them as integral parts of their systems (Kao et al., 2013; Yoshimoto et al., 2013; Xing et al., 2013; Lee and Lee, 2014; Junczys-Dowmunt and Grundkiewicz,…”
- Corpora Generation for Grammatical Error Correction (2019, methodology): “We report results on the CoNLL-2014 test set (Ng et al., 2014) and the JFLEG test set (Napoles et al.”
- Ground Truth for Grammatical Error Correction Metrics (2015, methodology): “task and borrowing metrics from machine translation, as Park and Levy (2011) did with BLEU (Papineni et al.”
- Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data (2019, methodology): “We test our network on two GEC benchmarks: the commonly used NUCLE CoNLL 2014 task [43], and the AESW dataset [44].”
- JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction (2017, methodology): “The CoNLL Shared Tasks used this data (Ng et al., 2014; Ng et al., 2013), and the 1,312 sentence test set from the 2014 task has become de rigueur for benchmarking GEC .”
- Lossless Acceleration for Seq2seq Generation with Aggressive Decoding (2022, methodology): “…evaluation, we follow the previous studies (Omelianchuk et al., 2020; Chen et al., 2020) which conduct GEC efficiency evaluation to use CoNLL-2014 (Ng et al., 2014) dataset that contains 1,312 sentences as our main test set, and evaluate the speedup as well as Max-Match (Dahlmeier & Ng, 2012)…”
- Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting (2019, methodology): “5 (Dahlmeier and Ng, 2012) for CoNLL-2014 and GLEU (Napoles et al., 2015) for JFLEG evaluation.”
- The BEA-2019 Shared Task on Grammatical Error Correction (2019, methodology): “This is significant because systems have traditionally only been tested on the CoNLL-2014 test set, which only contains 50 essays (1,312 sentences) on 2 different topics written by 25 South-East Asian undergraduates (Ng et al., 2014).”
- Artificial error generation for translation-based grammatical error correction (2016, methodology): “The second edition of this shared task (Ng et al., 2014) extended correction to all 28 types in updated versions of the training and test data.”
- Erroneous data generation for Grammatical Error Correction (2019, methodology): “Then a hybrid PBMT-NMT sys-tem (Grundkiewicz and Junczys-Dowmunt, 2018) appeared to reach the new state-of-the-art on the CoNLL-2014 benchmark.”
- A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction (2018, background): “GEC gained much attention within the NLP community after the CoNLL-2014 shared task (Ng et al. 2014) was organized.”
- Generating bug-fixes using pretrained transformers (2021, methodology): “Drawing from the CoNLL GEC leaderboard (Ng et al., 2014), we see several different approaches using transformers.”
- Encode, Tag, Realize: High-Precision Text Editing (2019, background): “We evaluate our method by conducting experiments on four different text editing tasks: Sentence Fusion, Split and Rephrase, Abstractive Summarization, and Grammatical Error Correction.”
- Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation (2020, methodology): “Experiments are conducted on the following widely used benchmarks: CoNLL-2014 (Ng et al., 2014), FCE (Yannakoudakis et al., 2011), BEA-2019 (Bryant et al., 2019).”