Neural Data-to-Text Generation with LM-based Text Augmentation

By Ernie Chang, Xiaoyu Shen, D. Zhu, Vera Demberg, Hui Su. 2021

46 citations recorded by the source corpus.

For many new application domains for data-to-text generation, the main obstacle in training neural models consists of a lack of training data. While usually large numbers of instances are available on the data side, often only very few text samples are available. To address this problem, we here propose a novel few-shot approach for this setting. Our approach automatically augments the data available for training by (i) generating new text samples based on replacing specific values by alternative ones from the same category, (ii) generating new text samples based on GPT-2, and (iii) proposing an automatic method for pairing the new text samples with data samples. As the text augmentation can introduce noise to the training data, we use cycle consistency as an objective, in order to make sure that a given data sample can be correctly reconstructed after having been formulated as text (and that text samples can be reconstructed from data). On both the E2E and WebNLG benchmarks, we show that this weakly supervised training paradigm is able to outperform fully supervised sequence-to-sequence models with less than 10% of the training set. By utilizing all annotated data, our model can boost the performance of a standard sequence-to-sequence model by over 5 BLEU points, establishing a new state-of-the-art on both datasets.

Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Natural Language Generation

Sebastian Gehrmann

Citations in the corpus

Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, methodology): “More recently, Chang et al. (2021) have developed a method for randomly generating new text samples with GPT-2 then automatically pairing them with data samples.”
A Survey on Data Augmentation for Text Classification (2021, unknown): “[118] propose methods using generative models for tasks other than text classification.”
Building Adaptive Acceptability Classifiers for Neural NLG (2021, methodology): “More recently, Chang et al. (2021) have developed a method for randomly generating new text samples with GPT-2 then automatically pairing them with data samples.”
Getting to Production with Few-shot Natural Language Generation Models (2021, methodology): “More recently, Chang et al. (2021) have developed a method for randomly generating new text samples with GPT-2 then automatically pairing them with data samples.”
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets (2021, background): “[11] achieve new state-of-the-art results on E2E and WebNLG benchmarks by using GPT-2 to augment training data.”