Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision

By Wanyu Du, Zae Myung Kim, Vipul Raheja, Dhruv Kumar, Dongyeop Kang. 2022

72 citations recorded by the source corpus.

Revision is an essential part of the human writing process. It tends to be strategic, adaptive, and, more importantly, iterative in nature. Despite the success of large language models on text revision tasks, they are limited to non-iterative, one-shot revisions. Examining and evaluating the capability of large language models for making continuous revisions and collaborating with human writers is a critical step towards building effective writing assistants. In this work, we present a human-in-the-loop iterative text revision system, Read, Revise, Repeat (R3), which aims at achieving high quality text revisions with minimal human efforts by reading model-generated revisions and user feedbacks, revising documents, and repeating human-machine interactions. In R3, a text revision model provides text editing suggestions for human writers, who can accept or reject the suggested edits. The accepted edits are then incorporated into the model for the next iteration of document revision. Writers can therefore revise documents iteratively by interacting with the system and simply accepting/rejecting its suggested edits until the text revision model stops making further revisions or reaches a predefined maximum number of revisions. Empirical experiments show that R3 can generate revisions with comparable acceptance rate to human writers at early revision depths, and the human-machine interaction can get higher quality revisions with fewer iterations and edits. The collected human-model interaction dataset and system code are available at https://github.com/vipulraheja/IteraTeR. Our system demonstration is available at https://youtu.be/lK08tIpEoaE.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Self-Refine: Iterative Refinement with Self-Feedback (2023, methodology): “Pairs of feedback and refinement have been used to learn supervised refiners (Schick et al., 2022b; Du et al., 2022; Yasunaga and Liang, 2020; Madaan et al., 2021).”
PEER: A Collaborative Language Model (2022, methodology): “For this evaluation, we consider the following datasets: • JFLEG (Napoles et al., 2017) is a grammatical error correction dataset with singlesentence inputs written by English language learners; • ASSET (Alva-Manchego et al., 2020) is a crowdsourced corpus for single-sentence text simplification; • ”
Help me write a Poem - Instruction Tuning as a Vehicle for Collaborative Poetry Writing (2022, background): “, 2022; Branwen, 2020) such as building co-creative assistants for writing stories, poems, and argumentative essays (Lee et al., 2022; Swanson et al., 2021; Uthus et al., 2019; Donahue et al., 2020; Padmakumar and He, 2022; Du et al., 2022).”
Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals (2022, background): “Some studies consulted with expert linguists [31], university students in the sciences [42] or humanities [123], and amateur writers [2, 21, 62, 100, 125].”