Adapting Sequence Models for Sentence Correction

By Allen Schmaltz, Yoon Kim, Alexander M. Rush, Stuart M. Shieber. 2017

54 citations recorded by the source corpus.

In a controlled experiment of sequence-to-sequence approaches for the task of sentence correction, we find that character-based models are generally more effective than word-based models and models that encode subword information via convolutions, and that modeling the output data as a series of diffs improves effectiveness over standard approaches. Our strongest sequence-to-sequence model improves over our strongest phrase-based statistical machine translation model, with access to the same data, by 6 M2 (0.5 GLEU) points. Additionally, in the data environment of the standard CoNLL-2014 setup, we demonstrate that modeling (and tuning against) diffs yields similar or better M2 scores with simpler models and/or significantly less data than previous sequence-to-sequence approaches.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Retrieve and Refine: Improved Sequence Generation Models For Dialogue (2018, background): “…(Junczys-Dowmunt and Grundkiewicz, 2017; Niehues et al., 2016; Novak et al., 2016; Xia et al., 2017; Grangier and Auli, 2017) or combining with retrieval (Gu et al., 2017; Cao et al., 2018), as well as for sentence correction or refinement without context (Guu et al., 2017; Schmaltz et al., 2017).”
Parallel Iterative Edit Models for Local Sequence Transduction (2019, methodology): “Schmaltz et al. (2017) interleave edit tags in target tokens but use seq2seq learning to predict the output sequence.”
Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data (2019, background): “Schmaltz et al. [6] used a word-level bidirectional LSTM network.”
Learning to Repair Software Vulnerabilities with Generative Adversarial Networks (2018, background): “Neural Machine Translation (NMT) systems have recently achieved the state-of-the-art performance on language translation and correction tasks [3, 4, 5, 6].”
Neural Networks for Text Correction and Completion in Keyboard Decoding (2017, background): “The work by Schmaltz et al [35], performs grammatical corrections in sentences using a complete character based sequence to sequence model.”
Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study (2018, methodology): “, 2014) for grammatical error correction (GEC) have drawn growing attention (Yuan & Briscoe, 2016; Xie et al., 2016; Ji et al., 2017; Schmaltz et al., 2017; Sakaguchi et al., 2017; Chollampatt & Ng, 2018; JunczysDowmunt et al., 2018) in recent years.”
Neural Networks for Modeling Source Code Edits (2018, background): “ation about past edits that is propagated forward is the resulting state after applying the edits, which will not generally contain enough information to disambiguate the intent of the edit sequence. Schmaltz et al. (2017) studies the problem of grammar correction. While the use of diff-like concept”
Fluency Boost Learning and Inference for Neural Grammatical Error Correction (2018, methodology): “• Adapt-seq2seq: a seq2seq model adapted to incorporate edit operations (Schmaltz et al., 2017).”
CoCoNuT: combining context-aware neural translation models using ensemble for program repair (2020, methodology): “Grammatical Error Correction (GEC): Recent work uses machine translation to fix grammar errors [14, 15, 18, 19, 30, 32, 51, 63, 72, 74, 90].”
A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction (2018, methodology): “More recently, Schmaltz et al. (2017) used a word-level bidirectional LSTM network trained on Lang-8 and NU-CLE (1.4M sentence pairs) with edit operations (insertions, deletions, and substitutions) marked with special tags in the target sentences.”