Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs

By Kuan-Hao Huang, Kai-Wei Chang. 2021

82 citations recorded by the source corpus.

Paraphrase generation plays an essential role in natural language process (NLP), and it has many downstream applications. However, training supervised paraphrase models requires many annotated paraphrase pairs, which are usually costly to obtain. On the other hand, the paraphrases generated by existing unsupervised approaches are usually syntactically similar to the source sentences and are limited in diversity. In this paper, we demonstrate that it is possible to generate syntactically various paraphrases without the need for annotated paraphrase pairs. We propose Syntactically controlled Paraphrase Generator (SynPG), an encoder-decoder based model that learns to disentangle the semantics and the syntax of a sentence from a collection of unannotated texts. The disentanglement enables SynPG to control the syntax of output paraphrases by manipulating the embedding in the syntactic space. Extensive experiments using automatic metrics and human evaluation show that SynPG performs better syntactic control than unsupervised baselines, while the quality of the generated paraphrases is competitive. We also demonstrate that the performance of SynPG is competitive or even better than supervised models when the unannotated data is large. Finally, we show that the syntactically controlled paraphrases generated by SynPG can be utilized for data augmentation to improve the robustness of NLP models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tailor: Generating and Perturbing Text with Semantic Controls (2021, background): “…incorporating syntactic structures for paraphrasing (Iyyer et al., 2018; Chen et al., 2019; Bao et al., 2019; Kumar et al., 2020; Sun et al., 2021; Huang and Chang, 2021) or discrete semantic signatures for diverse generation (Weir et al., 2020) are similar to T ailor in their high-dimensional…”
Impossible Distillation for Paraphrasing and Summarization: How to Make High-quality Lemonade out of Small, Low-quality Model (2023, background): “…Conventional approaches for unsupervised paraphrasing and summarization have focused on task-specific surrogates – e.g., back-translation (Wi-eting and Gimpel, 2018; Hu et al., 2019b) and autoencoding (Huang and Chang, 2021; Baziotis et al., 2019) – that guide the model toward desired output.”
AESOP: Paraphrase Generation with Adaptive Syntactic Control (2021, methodology): “We generate 5 paraphrases using selected templates in SynPG.”