Generating Sentences from Disentangled Syntactic and Semantic Spaces

By Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xinyu Dai, Jiajun Chen. 2019

122 citations recorded by the source corpus.

Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space. However, generating sentences from the continuous latent space does not explicitly model the syntactic information. In this paper, we propose to generate sentences from disentangled syntactic and semantic spaces. Our proposed method explicitly models syntactic information in the VAE’s latent space by using the linearized tree sequence, leading to better performance of language generation. Additionally, the advantage of sampling in the disentangled syntactic and semantic latent spaces enables us to perform novel applications, such as the unsupervised paraphrase generation and syntax transfer generation. Experimental results show that our proposed model achieves similar or better performance in various tasks, compared with state-of-the-art related work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tailor: Generating and Perturbing Text with Semantic Controls (2021, background): “Recent works incorporating syntactic structures for paraphrasing (Iyyer et al., 2018; Chen et al., 2019; Bao et al., 2019; Kumar et al., 2020; Sun et al., 2021; Huang and Chang, 2021) or discrete semantic signatures for diverse generation (Weir et al., 2020) are similar to T ailor in their…”
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, background): “…are often disentangled into sentiment (Fu et al., 2017; John et al., 2019), content (Romanov et al., 2019; Bao et al., 2019) and syntax (Bao et al., 2019) information through supervised learning from pre-deﬁned labels (John et al., 2019) or unsupervised learning with adversarial…”
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models (2020, background): “Novelty cutoffs do not depend on the reference, only the source, and are equivalent to selecting with BLEU-ori (Novelty is 100 − BLEU-ori) by Miao et al. (2019) or Bao et al. (2019).”
Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs (2021, background): “Chen et al. (2019b,a); Bao et al. (2019) disentangle the semantics and the syntax of a sentence.”
On Variational Learning of Controllable Representations for Text without Supervision (2019, methodology): “In addition, we model the relevant factors that we wish to control separated from the irrelevant factors by splitting z into two parts, z(1) and z(2), following prior work (Bao et al., 2019).”
Disentangled Representation Learning for Non-Parallel Text Style Transfer (2018, background): “Bao et al. (2019) have successfully shown that the syntax and semantics of a sentence can be disentangled from each other.”
Synthetic Data for Deep Learning (2019, background): “[709] and has been extended into text style transfer by disentangling style and content in the latent space [285], disentangling syntax and semantics [36], DialogWAE for dialog modeling [215], Bilingual-GAN able to generate parallel sentences in two languages [479], and other works.”
Variational Template Machine for Data-to-Text Generation (2020, methodology): “Recent works explored methods to learn disentangled latent variables (Hu et al., 2017a; Zhou & Neubig, 2017; Bao et al., 2019).”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “As a further step, we want to ensure that the content information is exclusively captured in z, namely, not contained in a at all, via the following AdvBow loss on a (John et al. 2019; Bao et al. 2019).”
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning (2020, background): “Unsupervised paraphrasing requires repeating this information (Miao et al., 2019; Bao et al., 2019), as does translation, but with a bilingual transformation (Artetxe et al., 2017; Lample et al., 2018).”