Style Transfer Through Back-Translation

By Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, Alan W. Black. 2018

410 citations recorded by the source corpus.

Style transfer is the task of rephrasing the text to contain specific stylistic properties without changing the intent or affect within the context. This paper introduces a new method for automatic style transfer. We first learn a latent representation of the input sentence which is grounded in a language translation model in order to better preserve the meaning of the sentence while reducing stylistic properties. Then adversarial generation techniques are used to make the output match the desired style. We evaluate this technique on three different style transformations: sentiment, gender and political slant. Compared to two state-of-the-art style transfer modeling techniques we show improvements both in automatic evaluation of style transfer and in manual evaluation of meaning preservation and fluency.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Self-Refine: Iterative Refinement with Self-Feedback (2023, background): “While a large body of work on style transfer is directed at sentence-level sentiment transfer (Li et al., 2018; Prabhumoye et al., 2018), we focus on transferring the sentiment of entire reviews, making the task challenging and providing opportunities for iterative improvements.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, neutral): “…summarization, data-to-text generation, and stylistic re-writing, among others) (Raffel et al., 2020; Zhang et al., 2020b; Aghajanyan et al., 2022; Prabhumoye et al., 2018), (2) or open-ended tasks (such as dialogue generation, prompt-based autocompletion, story generation, and more) (Adiwardana…”
How Can We Know What Language Models Know? (2019, methodology): “phrasing, we follow the simple method of using back-translation (Prabhumoye et al., 2018) to first translate the initial prompt into B candidates in another language, each of which is then back-translated into B candidates in the original language.”
A Recipe for Arbitrary Text Style Transfer with Large Language Models (2021, methodology): “Other Acc BLEU PPL S UPERVISED M ETHODS Cross-alignment (Shen et al., 2017) 73.4 17.6 812 Backtrans (Prabhumoye et al., 2018) 90.5 5.1 424 Multidecoder (Fu et al., 2018) 50.3 27.7 1,703 Delete-only (Li et al., 2018) 81.4 28.6 606 Delete-retrieve (Li et al., 2018) 86.2 31.1 948 Unpaired RL (Xu et al.”
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “While PLMs have demonstrated remarkable text generation and understanding capability, they can come with potential risks or harms [2, 3, 5] such as generating misinformation [46] or amplifying harmful biases [49].”
Politeness Transfer: A Tag and Generate Approach (2020, methodology): “, 2018), gender and political slant transfer (Reddy and Knight, 2016; Prabhumoye et al., 2018), and formality transfer (Rao and Tetreault, 2018; Xu et al.”
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout (2019, methodology): “Back-translation Back translation (Sennrich et al., 2016), a popular semi-supervised learning method, has been well studied in neural machine translation (Hoang et al., 2018; Wang et al., 2018b; Edunov et al., 2018; Prabhumoye et al., 2018).”
Conditional BERT Contextual Augmentation (2018, background): “Style transfer is deﬁned as the task of rephrasing the text to contain speciﬁc stylistic properties without changing the intent or affect within the context(Prabhumoye et al., 2018).”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, methodology): “BST We compare to the backtranslation style transfer model from Prabhumoye et al. (2018).”
Style Transfer as Unsupervised Machine Translation (2018, neutral): “Many approaches have been proposed to build style transfer systems and achieve promising performance (Hu et al. 2017; Shen et al. 2017; Fu et al. 2018; Li et al. 2018; Prabhumoye et al. 2018).”
Complementary Auxiliary Classifiers for Label-Conditional Text Generation (2020, methodology): “On the Yelp dataset, we compare with additional previous methods, including Cross-Align (Shen et al. 2017), MultiDecoder (Fu et al. 2018), DeleteAndRetrieve (Li et al. 2018), StyleTransformer (Dai et al. 2019), Back-Translation (Prabhumoye et al. 2018), and iVAE MI (Le Fang 2019).”
Gradient-based Constrained Sampling from Language Models (2022, background): “∇ E + 2 η ( t − 1) βz ( t 1) ) ; Update et al., 2019) or fine-tuning (Gururangan et al., 2020; Chan et al., 2021) for prompt based generation and generative models for tasks such for style transfer (Lample et al., 2019; Ziegler et al., 2020; Prabhumoye et al., 2018; Yu et al., 2017).”
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020, methodology): “We use the generated outputs for the Formality test set available in the public repository of Luo et al. (2019) (including outputs from the algorithms described in Prabhumoye et al., 2018 and Li et al., 2018) and run them on our evaluation pipeline.”
Substructure Substitution: Structured Data Augmentation for NLP (2021, background): “…them is back-translation (Sennrich et al., 2016), which has been demonstrated effective on not only machine translation, but also style-transfer (Prabhumoye et al., 2018; Zhang et al., 2020a), conditional text generation (Sobrevilla Cabezudo et al., 2019), and grammatical error correction (Xie…”
Text Detoxification using Large Pre-trained Neural Models (2021, background): “Machine Translation There is evidence that automatic translation tends to eliminate toxicity (Prabhumoye et al., 2018).”
Multiple-Attribute Text Rewriting (2018, methodology): “…dos Santos et al., 2018; Zhang et al., 2018c) requires methods such as REINFORCE (He et al., 2016) or approximating the output softmax layer with a tunable temperature (Hu et al., 2017; Prabhumoye et al., 2018; Yang et al., 2018), all of which tend to be slow, unstable and hard to tune in practice.”
(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas (2019, background): “Prabhumoye et al. (2018) proposes back-translation to get a style-agnostic sentence representation.”
Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation (2019, methodology): “Most of the previous methods (Hu et al., 2017; Shen et al., 2017; Fu et al., 2018; Carlson et al., 2017; Zhang et al., 2018b,a; Prabhumoye et al., 2018; Jin et al., 2019; Melnyk et al., 2017; dos Santos et al., 2018) formulate the style transfer problem into the “encoder-decoder” framework.”