FELIX: Flexible Text Editing Through Tagging and Insertion

By Jonathan Mallinson, Aliaksei Severyn, Eric Malmi, Guillermo Garrido. 2020

89 citations recorded by the source corpus.

We present FELIX – a flexible text-editing approach for generation, designed to derive maximum benefit from the ideas of decoding with bi-directional contexts and self-supervised pretraining. In contrast to conventional sequenceto-sequence (seq2seq) models, FELIX is efficient in low-resource settings and fast at inference time, while being capable of modeling flexible input-output transformations. We achieve this by decomposing the text-editing task into two sub-tasks: tagging to decide on the subset of input tokens and their order in the output text and insertion to in-fill the missing tokens in the output not present in the input. The tagging model employs a novel Pointer mechanism, while the insertion model is based on a Masked Language Model (MLM). Both of these models are chosen to be non-autoregressive to guarantee faster inference. FELIX performs favourably when compared to recent text-editing methods and strong seq2seq baselines when evaluated on four NLG tasks: Sentence Fusion, Machine Translation Automatic Post-Editing, Summarization, and Text Simplification

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

PEER: A Collaborative Language Model (2022, background): “, 2022) and explore more efficient ways of representing edits (Mallinson et al., 2020).”
Learning to Model Editing Processes (2022, background): “Furthermore, with text style transfer in which the difference between sentences in different styles (e.g. positive vs negative) can be relatively minimal (Reid and Zhong, 2021; Mallinson et al., 2020).”
Understanding Iterative Revision from Human-Written Text (2022, methodology): “Following prior works (Malmi et al., 2019; Dong et al., 2019; Mallinson et al., 2020), we report SARI, BLEU, and ROUGE-L”
Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020, methodology): “A more recent line of research frames sequence editing as a labelling problem using labels such as ADD, KEEP, and DELETE (Ribeiro et al., 2018; Dong et al., 2019; Mallinson et al., 2020; Malmi et al., 2019; Awasthi et al., 2019), often relying heavily on BERT (Devlin et al., 2019) pre-training.”
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021, methodology): “Text editing models (Malmi et al., 2019; Dong et al., 2019b; Mallinson et al., 2020) cast text generation as a sequence tagging problem with carefully selected edit operations required for the task.”