Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning

By Dayiheng Liu, Jie Fu, Yidan Zhang, C. Pal, Jiancheng Lv. 2019

53 citations recorded by the source corpus.

Typical methods for unsupervised text style transfer often rely on two key ingredients: 1) seeking the explicit disentanglement of the content and the attributes, and 2) troublesome adversarial learning. In this paper, we show that neither of these components is indispensable. We propose a new framework that utilizes the gradients to revise the sentence in a continuous space during inference to achieve text style transfer. Our method consists of three key components: a variational auto-encoder (VAE), some attribute predictors (one for each attribute), and a content predictor. The VAE and the two types of predictors enable us to perform gradient-based optimization in the continuous space, which is mapped from sentences in a discrete space, to find the representation of a target sentence with the desired attributes and preserved content. Moreover, the proposed method naturally has the ability to simultaneously manipulate multiple fine-grained attributes, such as sentence length and the presence of specific words, when performing text style transfer tasks. Compared with previous adversarial learning based methods, the proposed method is more interpretable, controllable and easier to train. Extensive experimental studies on three popular text style transfer tasks show that the proposed method significantly outperforms five state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

A Recipe for Arbitrary Text Style Transfer with Large Language Models (2021, methodology): “Such methods include latent representation manipulation (Liu et al., 2020), prototype-based text editing (Li et al., 2018), and pseudo-parallel corpus construction (Jin et al., 2019).”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “VAE is also commonly used in TST works (Mueller, Gifford, and Jaakkola 2017; Hu et al. 2017; Liu et al. 2020; Liao et al. 2018; Yi et al. 2020; Tikhonov et al. 2019).”
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, methodology): “However, the t-SNE visualization of the latent space in Liu et al. (2019b) allowed us to assume that encoders can output a latent representation z , attending to content rather than on an attribute, with a similar training.”
Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles (2019, background): “…graded style transfer by editing the hidden representation of an input towards one that would be classiﬁed more readily into a target style (Wang et al., 2019; Liu et al., 2019), or sampling responses around a given output to select those that better match a target style (Gao et al., 2019).”