Unsupervised Text Style Transfer using Language Models as Discriminators

By Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick. 2018

278 citations recorded by the source corpus.

Binary classifiers are often employed as discriminators in GAN-based unsupervised style transfer systems to ensure that transferred sentences are similar to sentences in the target domain. One difficulty with this approach is that the error signal provided by the discriminator can be unstable and is sometimes insufficient to train the generator to produce fluent language. In this paper, we propose a new technique that uses a target domain language model as the discriminator, providing richer and more stable token-level feedback during the learning process. We train the generator to minimize the negative log likelihood (NLL) of generated sentences, evaluated by the language model. By using a continuous approximation of discrete sampling under the generator, our model can be trained using back-propagation in an end- to-end fashion. Moreover, our empirical results show that when using a language model as a structured discriminator, it is possible to forgo adversarial steps during training, making the process more stable. We compare our model with previous work using convolutional neural networks (CNNs) as discriminators and show that our approach leads to improved performance on three tasks: word substitution decipherment, sentiment modification, and related language translation.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “…approach that uses the target-to-source model to generate noisy sources and then trains the source-to-target model to reconstruct the targets (also related to cycle-consistency (Zhu et al., 2017; Hoffman et al., 2018) and style transfer (Shen et al., 2017; Yang et al., 2018; Zhang et al., 2019)).”
Towards Coherent and Cohesive Long-form Text Generation (2018, methodology): “Thus, reward shaping was unnecessary, unlike previous works (Li et al., 2017; Yang et al., 2018) that further provided signals for partially generated sequences.”
A Distributional Approach to Controlled Text Generation (2020, methodology): “Yang et al. (2018) use a set of Language Models pretrained on the target domain as a control signal for text style transfer.”
Politeness Transfer: A Tag and Generate Approach (2020, background): “Current style transfer techniques (Shen et al., 2017; Hu et al., 2017; Fu et al., 2018; Yang et al., 2018; John et al., 2019) try to disentangle source style from content and then combine the content with the target style to generate the sentence in the target style.”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, background): “In unsupervised settings, a majority of work has dealt with the dearth of parallel data by using encoderdecoder setups paired with discriminators to disentangle style from content and steer generations (e.g., Shen et al., 2017; Zhang et al., 2018; Fu et al., 2018; Yang et al., 2018; Niu and Bansal, ”
Blank Language Models (2020, methodology): “Metrics We use evaluation methods introduced by prior work (Shen et al., 2017; Li et al., 2018; Wu et al., 2019; Yang et al., 2018).”
CoCon: A Self-Supervised Approach for Controlled Text Generation (2020, background): “Adversarial Loss Adversarial training objectives have shown to help in generating realistic text outputs [33].”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “(2021) showed that METEOR and WMD have better correlation with human evaluation than BLEU, although, in practice, BLEU is the most widely used metric to evaluate the semantic similarity between the source sentence and style-transferred output (Yang et al. 2018; Madaan et al. 2020).”
Generative Pretraining for Paraphrase Evaluation (2021, methodology): “…learning model that predicts a 082 conditioning factor for sequence-to-sequence para-083 phrase generation as one of its pretraining objec-084 tives, inspired by style transfer in text-to-speech085 (Skerry-Ryan et al., 2018) and text generation sys-086 tems (Yang et al., 2018; Lample et al., 2018).”
Fair Transfer of Multiple Style Attributes in Text (2019, background): “More recent neural network techniques such as variational autoencoders (VAEs) and generative adversarial networks (GANs) are designed to work with non-parallel vision datasets and are not easily extensible due to the discrete nature of the text [14], [16].”
A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer (2019, background): “[ Shen et al. , 2017; Hu et al. , 2017; Yang et al. , 2018b ] combine Variational Auto-encoder (VAE) with a style discriminator.”
Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020, background): “In this way, the dialogue policy and reward function are iteratively optimized, which requires great training efforts and might lead to unstable learning results (Yang et al., 2018).”
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, methodology): “We follow the current approach of the community (Yang et al., 2018; Logeswaran et al., 2018; Wang et al., 2019; Xu et al., 2019a; Lample et al., 2019; Dai et al., 2019a; He et al., 2020) and approximate the three criteria with the following metrics: 1.”
TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling (2021, background): “Typically, labels are assumed to be available for both source and target styles (Dai et al., 2019; He et al., 2020; Lample et al., 2019; Li et al., 2018; Niu et al., 2018; Prabhumoye et al., 2018; Shen et al., 2017; Sudhakar et al., 2019; Wu et al., 2019; Yang et al., 2018).”
A Survey on Computational Propaganda Detection (2020, background): “Zellers et al. [ 2019 ] showed that a template system helps manipulate the output format of a language model, while Yang et al. [ 2018 ] suggested how to transfer the style of the language model to the target domain.”
On Variational Learning of Controllable Representations for Text without Supervision (2019, methodology): “e models, such as Variational Autoencoders (VAEs) [10], Generative Adversarial Nets (GANs) [5], and autoregressive models [12], existing models have made progress towards controllable text generation [15, 8, 9, 18]. However, all these models rely on learning from annotated attributes to generate the”
Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning (2019, methodology): “Unlike most previous methods (Shen et al. 2017; Fu et al. 2018; Prabhumoye et al. 2018; Li et al. 2018; Yang et al. 2018) that only consider a single binary attribute (e.g., positive or negative sentiments), our approach naturally has the ability to control multiple ﬁne-grained attributes during…”
Adversarially Regularized Autoencoders (2017, result): “Finally, our work adds to the recent line of work on unaligned style transfer for text (Hu et al., 2017; Mueller et al., 2017; Li et al., 2018; Prabhumoye et al., 2018; Yang et al., 2018).”
Learning Dynamic Author Representations with Temporal Language Models (2019, background): “They are nowadays used at the core of an increasing number of tasks, for instance as a feature extractor for text classification [33], as a core building block of unsupervised Neural Machine Translation models [34], or as a discriminator for Generative Adversarial Models on text [35].”