Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models

By Raheel Qader, François Portet, Cyril Labbé. 2019

24 citations recorded by the source corpus.

In Natural Language Generation (NLG), End-to-End (E2E) systems trained through deep learning have recently gained a strong interest. Such deep models need a large amount of carefully annotated data to reach satisfactory performance. However, acquiring such datasets for every new NLG application is a tedious and time-consuming task. In this paper, we propose a semi-supervised deep learning scheme that can learn from non-annotated data and annotated data when available. It uses a NLG and a Natural Language Understanding (NLU) sequence-to-sequence models which are learned jointly to compensate for the lack of annotation. Experiments on two benchmark datasets show that, with limited amount of annotated data, the method can achieve very competitive results while not using any pre-processing or re-scoring tricks. These findings open the way to the exploitation of non-annotated datasets which is the current bottleneck for the E2E NLG system development to new applications.

Subfields: Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Natural Language Generation

Citations in the corpus

Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, methodology): “Most approaches to self-training for NLG— including earlier work on automatic data cleaning— make use of cycle consistency between parsing and generation models (Chisholm et al., 2017; Nie et al., 2019; Kedzie and McKeown, 2019; Qader et al., 2019).”
Neural Data-to-Text Generation with LM-based Text Augmentation (2021, methodology): “In comparison, Qader et al. (2019), though with access to all text samples at all percentages, still un-derperforms our model with tangible margin.”
Building Adaptive Acceptability Classifiers for Neural NLG (2021, methodology): “Regarding our self-training experiments, we note that self-training has been previously investigated for NLG by Kedzie and McKeown (2019), Qader et al. (2019) and Stevens-Guille et al. (2020), though they do not explore using pre-trained models with self-training.”
AggGen: Ordering and Aggregating while Generating (2021, background): “The prevailing sequence-to-sequence (seq2seq) architectures typically address this issue via reranking (Wen et al., 2015a; Dušek and Jurčı́ček, 2016; Juraska et al., 2018) or some sophisticated training techniques (Nie et al., 2019; Kedzie and McKeown, 2019; Qader et al., 2019).”
Getting to Production with Few-shot Natural Language Generation Models (2021, background): “Self-training has been previously investigated for NLG by Kedzie and McKeown (2019) and Qader et al. (2019), though they do not explore using pre-trained models with self-training.”