The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction

By Dimitrios Alikaniotis, Vipul Raheja. 2019

27 citations recorded by the source corpus.

Recent work on Grammatical Error Correction (GEC) has highlighted the importance of language modeling in that it is certainly possible to achieve good performance by comparing the probabilities of the proposed edits. At the same time, advancements in language modeling have managed to generate linguistic output, which is almost indistinguishable from that of human-generated text. In this paper, we up the ante by exploring the potential of more sophisticated language models in GEC and offer some key insights on their strengths and weaknesses. We show that, in line with recent results in other NLP tasks, Transformer architectures achieve consistently high performance and provide a competitive baseline for future machine learning models.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Luke S. Zettlemoyer
3. Mohit Bansal
4. Percy Liang
5. Yejin Choi
6. Graham Neubig
7. Jason Aaron Edward Weston
8. Yizhe Zhang

In Natural Language Processing

1. Michel Galley
2. Yizhe Zhang
3. Chris Brockett
4. Jianfeng Gao
5. Alec Radford
6. Bill Dolan
7. Zeqiu Wu
8. Jeff Wu

Citations in the corpus

• Release Strategies and the Social Impacts of Language Models (2019, background): “Grammarly published a paper highlighting GPT-2’s utility in grammatical error correction [2].”
• Automatic Document Sketching: Generating Drafts from Analogous Texts (2021, background): “Most end-user applications for assisting user writing, however, are confined to sentence-level generation (Chen et al., 2019; Kannan et al., 2016; Alikaniotis and Raheja, 2019; Prabhumoye et al., 2019; Faltings et al., 2021).”