Style Transfer from Non-Parallel Text by Cross-Alignment

By Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi S. Jaakkola. 2017

829 citations recorded by the source corpus.

This paper focuses on style transfer on the basis of non-parallel text. This is an instance of a broad family of problems including machine translation, decipherment, and sentiment modification. The key challenge is to separate the content from other aspects such as style. We assume a shared latent content distribution across different text corpora, and propose a method that leverages refined alignment of latent representations to perform style transfer. The transferred sentences from one style should match example sentences from the other style as a population. We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

The Curious Case of Neural Text Degeneration (2019, methodology): “(2) Computing the optimum argmax sequence from recurrent neural language models is not tractable, so consider two prominent decoding methods for approximating the argmax: Beam search is the most commonly used approximation in practice (Li et al., 2016c; Shen et al., 2017; Wiseman et al., 2017).”
Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “…approach that uses the target-to-source model to generate noisy sources and then trains the source-to-target model to reconstruct the targets (also related to cycle-consistency (Zhu et al., 2017; Hoffman et al., 2018) and style transfer (Shen et al., 2017; Yang et al., 2018; Zhang et al., 2019)).”
Generative Adversarial Networks (2021, neutral): “An overview of the cross-alignment method [12]”
Towards Understanding and Mitigating Social Biases in Language Models (2021, methodology): “Following human evaluation protocols in the related ﬁelds of text generation and style transfer (Shen et al., 2017), 5 annotators were asked to judge 1) clarity : coherence of the sentence (including grammar and spelling), 2) content : whether the sentence respects the semantic meaning of realistic…”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, background): “Shen et al. (2017); Hu et al. (2017) train variational auto-encoders for style transfer that rely on learning disentangled latent representations for style and content.”
A Recipe for Arbitrary Text Style Transfer with Large Language Models (2021, neutral): “Other Acc BLEU PPL S UPERVISED M ETHODS Cross-alignment (Shen et al., 2017) 73.4 17.6 812 Backtrans (Prabhumoye et al., 2018) 90.5 5.1 424 Multidecoder (Fu et al., 2018) 50.3 27.7 1,703 Delete-only (Li et al., 2018) 81.4 28.6 606 Delete-retrieve (Li et al., 2018) 86.2 31.1 948 Unpaired RL (Xu et al.”
Unsupervised Machine Translation Using Monolingual Corpora Only (2017, methodology): “A similar work to ours is the style transfer method with non-parallel text by Shen et al. (2017).”
Controllable Abstractive Summarization (2017, neutral): “(Shen et al., 2017) relies on adversarial training Set sizes”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “Some other works [59–66] focus on disentangling content from style.”
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks (2018, neutral): “Shen et al. (2017) perform style transfer on non-parallel texts, while Guu et al. (2017) generate novel sentences from prototypes; again, these methods are not necessarily seeking to generate meaning-preserving paraphrases, merely transformed sentences that have an altered style.”
Learning Multimodal Graph-to-Graph Translation for Molecular Optimization (2018, methodology): “Our adversarial training technique is inspired by recent text style transfer methods (Shen et al., 2017; Zhao et al., 2018) that adversarially regularize the continuous representation of discrete structures to enable end-to-end training.”
Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer (2018, methodology): “Previous work has attempted to use adversarial networks (Shen et al., 2017; Fu et al., 2018) for this task, but—as we demonstrate—their outputs tend to be low-quality, as judged by human raters.”
Towards Controllable Story Generation (2018, background): “, 2017), and different styles of text (Hu et al., 2017; Ficler and Goldberg, 2017; Shen et al., 2017; Fu et al., 2017).”
Politeness Transfer: A Tag and Generate Approach (2020, methodology): “Automated Evaluation Following prior work (Li et al., 2018; Shen et al., 2017), we use automatic metrics for evaluation of the models along two major dimensions: (1) style transfer accuracy and (2) content preservation.”
Style Transfer Through Back-Translation (2018, methodology): “We compare our model against the “cross-aligned” auto-encoder (Shen et al., 2017), which uses style-speciﬁc decoders to align the style of generated sentences to the actual distribution of the style.”
Style Transfer in Text: Exploration and Evaluation (2017, methodology): “Shen et al. (2017) explored style transfer for sentiment modiﬁcation, decipherment of word substitution ciphers and recovery of word order.”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, background): “…dearth of parallel data by using encoder-decoder setups paired with discriminators to disentangle style from content and steer generations (e.g., Shen et al., 2017; Zhang et al., 2018; Fu et al., 2018; Yang et al., 2018; Niu and Bansal, 2018; Romanov et al., 2019; Dai et al., 2019; John et al.,…”
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders (2018, methodology): “In this regard, cross-aligned VAE’s have been used previously for text-style transfer [25] and image-toimage translation [16].”