Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity

By Hamza Harkous, Isabel Groves, Amir Saffari. 2020

93 citations recorded by the source corpus.

End-to-end neural data-to-text (D2T) generation has recently emerged as an alternative to pipeline-based architectures. However, it has faced challenges generalizing to new domains and generating semantically consistent text. In this work, we present DataTuner, a neural, end-to-end data-to-text generation system that makes minimal assumptions about the data representation and target domain. We take a two-stage generation-reranking approach, combining a fine-tuned language model with a semantic fidelity classifier. Each component is learnt end-toe-nd without needing dataset-specific heuristics, entity delexicalization, or post-processing. We show that DataTuner achieves state of the art results on automated metrics across four major D2T datasets (LDC2017T10, WebNLG, ViGGO, and Cleaned E2E), with fluency assessed by human annotators as nearing or exceeding the human-written reference texts. Our generated text has better semantic fidelity than the state of the art on these datasets. We further demonstrate that our model-based semantic fidelity scorer is a better assessment tool compared to traditional heuristic-based measures of semantic accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Caiming Xiong
3. Percy Liang
4. Luke S. Zettlemoyer
5. Richard Socher
6. Hinrich Schütze
7. Jianfeng Gao
8. Graham Neubig

In Computer Vision and Pattern Recognition

1. Caiming Xiong
2. Luc Van Gool
3. Richard Socher
4. Mohit Bansal
5. Trevor Darrell
6. Dacheng Tao
7. Li Fei-Fei
8. Bernt Schiele

In Natural Language Processing

1. Mohit Bansal
2. Caiming Xiong
3. Richard Socher
4. Hinrich Schütze
5. Dragomir R. Radev
6. Mirella Lapata
7. Percy Liang
8. Luke S. Zettlemoyer

Citations in the corpus

• Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, background): “Additionally, semantic fidelity classifiers (Harkous et al., 2020) can be helpful in addressing issues with semantic correctness that are exacerbated in low-data settings (Anonymous, 2021).”
• DART: Open-Domain Structured Data Record to Text Generation (2020, methodology): “However, language models such as BART are pretrained by reconstructing text and, as a result, we found that their output on DART often contains hallucinated words (Parikh et al., 2020; Harkous et al., 2020; Reiter, 2020).”
• Investigating Pretrained Language Models for Graph-to-Text Generation (2020, methodology): “Recently, Harkous et al. (2020) and Kale (2020) demonstrate state-of-the-art re-sults in different data-to-text datasets, employing GPT-2 and T5 models respectively.”
• Pre-Trained Language Models for Text Generation: A Survey (2022, methodology): “[72]; Utilizing Copying Mechanism: Li et al.”
• Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference (2020, methodology): “Similarly to us, Harkous et al. (2020) use a pretrained neural model as a classifier to detect inaccurate output, finetuning the classifier on manually augmented domain-specific data.”
• Logic-Consistency Text Generation from Semantic Parses (2021, methodology): “2020) are not ideal for explicitly measuring the logic consistency (Wang et al., 2020b; Harkous et al., 2020), because they tend to evenly weight each word in the generated text without fully attending on the fatal logical keywords.”
• Building Adaptive Acceptability Classifiers for Neural NLG (2021, methodology): “We also compare against the techniques described in Harkous et al. (2020), which we call sentence transformation (SNT).”
• Control Prefixes for Parameter-Efficient Text Generation (2021, methodology): “The SOTA (Harkous et al., 2020) for E2E Clean consists of a fine-tuned GPT-2 with a semantic fidelity classifier trained on additional generated data.”
• Control Prefixes for Text Generation (2021, methodology): “CONTROL PREFIXES (A2), which can leverage the heterogeneous DART datasets, outperforms the SOTA in terms of BLEU.”
• Getting to Production with Few-shot Natural Language Generation Models (2021, methodology): “We explored several recent model-based acceptability checking mechanisms as alternatives (Harkous et al., 2020; Anonymous, 2021).”