Encode, Tag, Realize: High-Precision Text Editing

By Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, Aliaksei Severyn. 2019

191 citations recorded by the source corpus.

We propose LaserTagger - a sequence tagging approach that casts text generation as a text editing task. Target texts are reconstructed from the inputs using three main edit operations: keeping a token, deleting it, and adding a phrase before the token. To predict the edit operations, we propose a novel model, which combines a BERT encoder with an autoregressive Transformer decoder. This approach is evaluated on English text on four tasks: sentence fusion, sentence splitting, abstractive summarization, and grammar correction. LaserTagger achieves new state-of-the-art results on three of these tasks, performs comparably to a set of strong seq2seq baselines with a large number of training examples, and outperforms them when the number of examples is limited. Furthermore, we show that at inference time tagging can be more than two orders of magnitude faster than comparable seq2seq models, making it more attractive for running in a live environment.

Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Furu Wei
  2. Graham Neubig
  3. Mohit Bansal
  4. Luke S. Zettlemoyer
  5. Percy Liang
  6. Felix Stahlberg
  7. Jianfeng Gao
  8. Uri Alon

In Natural Language Processing

  1. Felix Stahlberg
  2. Graham Neubig
  3. Furu Wei
  4. Uri Alon
  5. Eran Yahav
  6. Tao Ge
  7. Miltiadis Allamanis
  8. Shankar Kumar

In Natural Language Generation

  1. Furu Wei
  2. Tao Ge
  3. Xin Sun
  4. Shankar Kumar
  5. Felix Stahlberg
  6. Houfeng Wang
  7. Heming Xia
  8. Zhuowei Liu

Citations in the corpus