A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning
By Yo Joong Choe, Jiyeon Ham, Kyubyong Park, Yeoil Yoon. 2019
89 citations recorded by the source corpus.
Grammatical error correction can be viewed as a low-resource sequence-to-sequence task, because publicly available parallel corpora are limited.To tackle this challenge, we first generate erroneous versions of large unannotated corpora using a realistic noising function. The resulting parallel corpora are sub-sequently used to pre-train Transformer models. Then, by sequentially applying transfer learning, we adapt these models to the domain and style of the test set. Combined with a context-aware neural spellchecker, our system achieves competitive results in both restricted and low resource tracks in ACL 2019 BEAShared Task. We release all of our code and materials for reproducibility.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Xiaojun Wan
- Mariano Felice
- Furu Wei
- Zhuowei Liu
- Varun Prashant Gangal
- Soroush Vosoughi
- Eduard H. Hovy
- Wenguang Wang
In Computer Vision and Pattern Recognition
- Max White
- Jason Wei
- Sarath Chandar
- Eduard H. Hovy
- Xiaojun Wan
- Steven Y. Feng
- Teruko Mitamura
- Soroush Vosoughi
In Natural Language Processing
- Mariano Felice
- Xiaojun Wan
- Varun Prashant Gangal
- Eduard H. Hovy
- Teruko Mitamura
- Soroush Vosoughi
- Zhuowei Liu
- Jason Wei
Citations in the corpus
- A Survey of Data Augmentation Approaches for NLP (2021, background): “Choe et al. (2019) learn error patterns from small annotated samples along with POS-specific noising.”
- Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation (2020, methodology): “Following Choe et al. (2019), we use spellcheck Enchant3 to assist the correction of spelling errors.”
- A Comprehensive Survey of Grammar Error Correction (2020, methodology): “Many Transformer based GEC systems adopted pre-training on artificial parallel GEC data [57], [56], [55], [99], [100].”
- The BEA-2019 Shared Task on Grammatical Error Correction (2019, methodology): “This is most likely because UEDIN-MS and Kakao&Brain both made effective use of artificial data.”
- A Comprehensive Survey of Grammatical Error Correction (2021, methodology): “generated artificial errors basing on inspection of frequent edits and error types in annotated GEC corpora [19].”
- A Comparative Study of Synthetic Data Generation Methods for Grammatical Error Correction (2020, methodology): “The method used in the UEDIN-MS system utilizes confusion sets generated by a spellchecker, while the Kakao&Brain method relies on learner patterns extracted from a small annotated sample and on POS-based confusions.”