Seq2Edits: Sequence Transduction Using Span-level Edit Operations

By Felix Stahlberg, Shankar Kumar. 2020

102 citations recorded by the source corpus.

We propose Seq2Edits, an open-vocabulary approach to sequence editing for natural language processing (NLP) tasks with a high degree of overlap between input and output texts. In this approach, each sequence-to-sequence transduction is represented as a sequence of edit operations, where each operation either replaces an entire source span with target tokens or keeps it unchanged. We evaluate our method on five NLP tasks (text normalization, sentence fusion, sentence splitting & rephrasing, text simplification, and grammatical error correction) and report competitive results across the board. For grammatical error correction, our method speeds up inference by up to 5.2x compared to full sequence models because inference time depends on the number of edits rather than the number of target tokens. For text normalization, sentence fusion, and grammatical error correction, our approach improves explainability by associating each edit operation with a human-readable tag.

Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Furu Wei
2. Yu Zhang
3. Ankur Bapna
4. Ye Jia
5. Alexis Conneau
6. Melvin Johnson
7. James R. Glass
8. Yu-An Chung

In Natural Language Processing

1. Furu Wei
2. Ankur Bapna
3. Yu Zhang
4. Ye Jia
5. Melvin Johnson
6. Alexis Conneau
7. Tao Ge
8. Graham Neubig

In Natural Language Generation

1. Xin Sun
2. Ye Jia
3. Houfeng Wang
4. Heming Xia
5. Zhuowei Liu
6. Furu Wei
7. Tao Ge

Citations in the corpus

• Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021, methodology): “Following GECToR ’s recipe, we leverage the pretrained model BART (Lewis et al., 2019) to initialize a 12+2 model which proves to work well in NMT (Li et al., 2021) despite more parameters, and apply the multi-stage fine-tuning strategy used in Stahlberg and Kumar (2020).”
• Lossless Acceleration for Seq2seq Generation with Aggressive Decoding (2022, methodology): “…et al., 2020) Transformer with a 12-layer encoder and 2-layer decoder that is pretrained with synthetic data and fine-tuned in multiple stages (Stahlberg & Kumar, 2020), and compares with state-of-the-art efficient GEC models that are all faster than the Transformer-big baseline in CoNLL-14…”
• A Comprehensive Survey of Grammatical Error Correction (2021, methodology): “Building on the previous work, Seq2Edits proposes span-based edits rather than token-level edits, which make the representation easier to learn [124].”
• SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training (2021, result): “In addition to GLUE, we also evaluate and report sentence error rate for text normalization and compare our approach to Stahlberg & Kumar (2020) in Table 4.”
• NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021, unknown): “) as a text editing task (Malmi et al., 2019; Panthaplackel et al., 2020; Stahlberg and Kumar, 2020)”