Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach
By Jingjing Xu, Xu Sun, Qi Zeng, Xuancheng Ren, Xiaodong Zhang, Houfeng Wang, Wenjie Li. 2018
215 citations recorded by the source corpus.
The goal of sentiment-to-sentiment “translation” is to change the underlying sentiment of a sentence while keeping its content. The main challenge is the lack of parallel data. To solve this problem, we propose a cycled reinforcement learning method that enables training on unpaired data by collaboration between a neutralization module and an emotionalization module. We evaluate our approach on two review datasets, Yelp and Amazon. Experimental results show that our approach significantly outperforms the state-of-the-art systems. Especially, the proposed method substantially improves the content preservation performance. The BLEU score is improved from 1.64 to 22.46 and from 0.56 to 14.06 on the two datasets, respectively.
Subfields: Artificial Intelligence, Reinforcement Learning, Sentiment & Opinion Analysis, Natural Language Generation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Percy Liang
- Yejin Choi
- Luke S. Zettlemoyer
- Graham Neubig
- Noah A. Smith
- Yoshua Bengio
In Reinforcement Learning
In Sentiment & Opinion Analysis
In Natural Language Generation
- Mohit Iyyer
- John Wieting
- Houfeng Wang
- Jianfeng Gao
- Zhifang Sui
- Baobao Chang
- Jie Zhou
- Sandeep Subramanian
Citations in the corpus
- A Recipe for Arbitrary Text Style Transfer with Large Language Models (2021, neutral): “…(Fu et al., 2018) 50.3 27.7 1,703 Delete-only (Li et al., 2018) 81.4 28.6 606 Delete-retrieve (Li et al., 2018) 86.2 31.1 948 Unpaired RL (Xu et al., 2018) 52.2 37.2 2,750 Dual RL (Luo et al., 2019) 85.9 55.1 982 Style transformer (Dai et al., 2019) to be a little less angsty • to be…”
- Conditional BERT Contextual Augmentation (2018, methodology): “In order to find style-relevant words in a sentence, we refer to Xu(Xu et al., 2018), which proposed an attention-based method to extract the contribution of each word to the sentence sentimental label.”
- Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020, methodology): “Specific implementations of the metrics have been provided in (Fu et al., 2018) 26.0 12.0 36.9 15.1 1.4 8.9 0.0 1.5 Del. (Li et al., 2018) 24.2 30.1 53.5 20.8 3.1 10.2 0.0 1.6 Unpaired (Xu et al., 2018) 53.9 1.6 16.3 34.9 0.4 10.9 0.0 2.2 DelRetri.”
- Deep Learning for Text Style Transfer: A Survey (2020, methodology): “Attention-based methods train an attribute classifier using the attention mechanism (Bahdanau, Cho, and Bengio 2015), and consider words with attention weights higher than average as markers (Xu et al. 2018).”
- Multiple-Attribute Text Rewriting (2018, background): “Most solutions use an adversarial approach to learn latent representations agnostic to the style of input sentences (Fu et al., 2017; Hu et al., 2017; Shen et al., 2017; Zhang et al., 2018a; Xu et al., 2018; John et al., 2018; Zhao et al., 2018).”
- Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, background): “Then, a decoder, aware of the destination attribute, generated a transferred sentence (Shen et al., 2017; Hu et al., 2017; Fu et al., 2018; Zhang et al., 2018c; Xu et al., 2018; John et al., 2019).”
- Keep Calm and Switch On! Preserving Sentiment and Fluency in Semantic Text Exchange (2019, unknown): “Notable works in style/sentiment transfer include (Shen et al., 2017; Fu et al., 2018; Li et al., 2018; Xu et al., 2018).”
- SGM: Sequence Generation Model for Multi-label Classification (2018, methodology): “, 2018), style transfer (Shen et al., 2017; Xu et al., 2018) and other domains, we propose a sequence generation model with a novel decoder structure to solve the MLC task.”
- Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory (2018, unknown): “For sentiment-to-sentiment translation, Xu et al. (2018) first use a neutralization module to remove emotional words and then add sentiment to the neutralized content.”
- Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation (2019, methodology): “…(Li et al., 2018) 85.5 9.7 28.6 79 N/A N/A N/A DeleteAndRetrieve (Li et al., 2018) (Fu et al., 2018) 49.9 9.2 37.9 127 N/A N/A N/A CycleRL(Xu et al., 2018) 88 Fluency Fluency is measured by the perplexity of the transferred sentence, and we trained a 5-gram language model on the training…”
- Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation (2018, methodology): “Recent researches have focused on developing methods to generate informative (Xu et al., 2018b) and diverse text (Li et al., 2017, 2016; Guu et al., 2017; Shao et al., 2017).”
- A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer (2019, methodology): “On the contrary, [ Li et al. , 2018; Zhang et al. , 2018a; Xu et al. , 2018 ] propose to separate content and style by directly removing the style words.”
- Complementary Auxiliary Classifiers for Label-Conditional Text Generation (2020, methodology): “Many methods for attribute transfer treat the problem as sequence-to-sequence translation, and source sentences must be provided in order to transfer labels in target sentences (Li et al. 2018; Xu et al. 2018; Zhang, Ding, and Soricut 2018).”
- Disentangled Representation Learning for Non-Parallel Text Style Transfer (2018, methodology): “05 Cycle-RL (Xu et al., 2018) ‡ 0.”
- Text Style Transfer: A Review and Experiment Evaluation (2020, methodology): “The scarcity of parallel data motivated a new breed of TST algorithms that attempt to transfer text style without parallel data [9, 11, 15, 18, 20, 22, 24, 29, 40, 41, 44, 46– 48, 54, 60, 67, 70, 72, 76, 78, 81, 82, 85, 86, 88, 90, 92, 93].”
- “Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer (2019, methodology): “These include reinforcement learning based approaches (Xu et al., 2018; Gong et al., 2019), an unsupervised machine translation based approach (Subramanian et al., 2018) and the DRG approach (Li et al., 2018).”
- Style Transfer as Unsupervised Machine Translation (2018, unknown): “Xu et al. 2018), in which attribute-related words are recognized and removed to generate a sentence containing only content information, which is used as a query to find a similar sentence with the target attribute from the corpus.”