A Recipe for Arbitrary Text Style Transfer with Large Language Models

By Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, Jason Wei. 2021

158 citations recorded by the source corpus.

In this paper, we leverage large language models (LLMs) to perform zero-shot text style transfer. We present a prompting method that we call augmented zero-shot learning, which frames style transfer as a sentence rewriting task and requires only a natural language instruction, without model fine-tuning or exemplars in the target style. Augmented zero-shot learning is simple and demonstrates promising results not just on standard style transfer tasks such as sentiment, but also on arbitrary transformations such as ‘make this melodramatic’ or ‘insert a metaphor.’

Subfields: Artificial Intelligence, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Natural Language Processing

Citations in the corpus

Finetuned Language Models Are Zero-Shot Learners (2021, background): “, 2020), as well as various NLP applications (Liu et al., 2019a; Corazza et al., 2020; Wang et al., 2021; Reif et al., 2021).”
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them (2022, background): “, 2022b), creative generation (Reif et al., 2022; Suzgun et al., 2022; Mittal et al., 2022), and answering questions about color, spatial, and cardinal concepts (Abdou et al.”
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (2021, background): “, in [3]), it lacks the capability to perform both tasks together well in an end-to-end manner.”
PAL: Program-aided Language Models (2022, background): “, 2021) has been a popular approach to tackle tasks ranging from text-generation (Gehrmann et al., 2021; Reif et al., 2021; Wei et al., 2021; Sanh et al., 2021) to code (Chen et al.”
Help me write a Poem - Instruction Tuning as a Vehicle for Collaborative Poetry Writing (2022, methodology): “Reif et al. (2022) present a prompting method which performs style transfer in a zero-shot or few-shot manner with only a natural language instruction describing the target style without model fine-tuning or exemplars in the target style.”
Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals (2022, background): “Recently, research has explored open-story generation using machine learning techniques which leverage large datasets, massive deep learning models, increased compute capacity, and large language model prompt engineering [11, 14, 38, 83, 89, 102].”
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models (2022, background): “…domain-conditioned neural language models (Prabhumoye et al., 2020; He et al., 2020; Lample et al., 2018; Shen et al., 2017; Krishna et al., 2020; Reif et al., 2021; Ficler and Goldberg, 2017; Khalifa et al., 2021) or ﬁnetun-ing/modifying an underlying large pre-trained base model for generation…”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, methodology): “For unsupervised tasks with more complex requirements such as text style transfer (Hu et al., 2017; Jin et al., 2022), Reif et al. (2021) proposed augmented zero-shot prompting, which achieves some success using huge LMs (e.g., GPT-3).”
How Many Data Samples is an Additional Instruction Worth? (2022, methodology): “They are allowed to use automated tools such as Semantic Control (Ross et al., 2021), Text Style Transfer (Reif et al., 2021), NL-Augmenter (Dhole et al., 2021).”
HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2022, methodology): “…(Wang et al., 2022c) such as dialog (Gupta et al., 2022), NER (Wang et al., 2022a), program synthesis (Kuznia et al., 2022), style transfer (Reif et al., 2021), tabular question answering (Luo et al., 2022), relation extraction (Chen et al., 2021), biomedical applications (Parmar et al.,…”
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models (2022, methodology): “, 2019), LLM (Reif et al., 2022), and our own method “Prompt-and-Rerank”, along with the ground-truth references.”