Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study

By Tao Ge, Furu Wei, M. Zhou. 2018

87 citations recorded by the source corpus.

Neural sequence-to-sequence (seq2seq) approaches have proven to be successful in grammatical error correction (GEC). Based on the seq2seq framework, we propose a novel fluency boost learning and inference mechanism. Fluency boosting learning generates diverse error-corrected sentence pairs during training, enabling the error correction model to learn how to improve a sentence's fluency from more instances, while fluency boosting inference allows the model to correct a sentence incrementally with multiple inference steps. Combining fluency boost learning and inference with convolutional seq2seq models, our approach achieves the state-of-the-art performance: 75.72 (F_{0.5}) on CoNLL-2014 10 annotation dataset and 62.42 (GLEU) on JFLEG test set respectively, becoming the first GEC system that reaches human-level performance (72.58 for CoNLL and 62.37 for JFLEG) on both of the benchmarks.

Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Natural Language Generation

Citations in the corpus

Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021, background): “2019; Lichtarge et al., 2019; Zhou et al., 2020; Wan et al., 2020), most methods that improve performance (Ge et al., 2018b; Kaneko et al., 2020) introduce additional computational cost and thus slow down inference despite the performance improvement.”
Parallel Iterative Edit Models for Local Sequence Transduction (2019, methodology): “Lichtarge et al. (2019) and Ge et al. (2018b) also reﬁne iteratively but with ED models.”
Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data (2019, methodology): “, 2018), involving rightto-left models (Ge et al., 2018b), or by pipelining SMT and NMT-based systems (Grundkiewicz and Junczys-Dowmunt, 2018).”
Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation (2020, methodology): “To address this problem, many data augmentation methods have been proposed (Ge et al., 2018).”
The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction (2019, unknown): “ Felice et al. (2014) † - - - 39.71 30.10 37.33 Yannakoudakis et al. (2017) - - - 58.79 30.63 49.66 Chollampatt and Ng (2017) - - - 62.74 32.96 53.14 Chollampatt and Ng (2018) - - - 65.49 33.14 54.79 Ge et al. (2018) - - - 74.12 36.30 61.34 Bryant and Briscoe (2018) 36.62 19.93 31.37 40.56 20.81 34.”
A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning (2019, methodology): “…al., 2018; Lichtarge et al., 2018; Zhao et al., 2019) made use of the Transformer (Vaswani et al., 2017) architecture and saw results nearly as good as or better than convolutional (Chollampatt and Ng, 2018a,b) and recurrent (Grundkiewicz and Junczys-Dowmunt, 2018; Ge et al., 2018a) architectures.”
CoCoNuT: combining context-aware neural translation models using ensemble for program repair (2020, methodology): “Grammatical Error Correction (GEC): Recent work uses machine translation to fix grammar errors [14, 15, 18, 19, 30, 32, 51, 63, 72, 74, 90].”
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction (2020, unknown): “…not only take time to rescore but also slow down the correction model with a larger beam size during inference; multi-round (iterative) decoding (Ge et al., 2018a,b; Lichtarge et al., 2019) needs to repeatedly run the model; BERT-fuse (Kaneko et al., 2020) adds extra computation for model…”
Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting (2019, methodology): “To address this challenge, we use ﬂuency discrimination , whose idea was ﬁrst proposed by Ge et al. (2018a).”
Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020, methodology): “This is very similar to the iterative decoding strategies used by Lichtarge et al. (2019); Awasthi et al. (2019); Ge et al. (2018); Grundkiewicz et al. (2019) with the difference that we pass through n-best lists between beam search passes rather than only the single best hypothesis.”