Neural data-to-text generation: A comparison between pipeline and end-to-end architectures

By Thiago Castro Ferreira, Chris van der Lee, Emiel van Miltenburg, Emiel Krahmer. 2019

151 citations recorded by the source corpus.

Traditionally, most data-to-text applications have been designed using a modular pipeline architecture, in which non-linguistic input data is converted into natural language through several intermediate transformations. By contrast, recent neural models for data-to-text generation have been proposed as end-to-end approaches, where the non-linguistic input is rendered in natural language with much less explicit intermediate representations in between. This study introduces a systematic comparison between neural pipeline and end-to-end data-to-text approaches for the generation of text from RDF triples. Both architectures were implemented making use of the encoder-decoder Gated-Recurrent Units (GRU) and Transformer, two state-of-the art deep learning methods. Automatic and human evaluations together with a qualitative analysis suggest that having explicit intermediate steps in the generation process results in better texts than the ones generated by end-to-end approaches. Moreover, the pipeline models generalize better to unseen inputs. Data and code are publicly available.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Natural Language Generation

Citations in the corpus

DART: Open-Domain Structured Data Record to Text Generation (2020, methodology): “Similar ﬁndings can be found in Castro Ferreira et al. (2019).”
Text-to-Text Pre-Training for Data-to-Text Tasks (2020, methodology): “• Pipeline-Transformer (Ferreira et al., 2019), a pipelined neural system consisting of discourse ordering, text structuring, lexicalization and referring expression generation.”
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020, methodology): “Our model is also in spirit related to recent attempts at separating content planning and surface realization in neural data-to-text models (Zhao et al., 2018; Puduppully et al., 2019; Moryossef et al., 2019; Ferreira et al., 2019).”
Data-to-text Generation with Macro Planning (2021, background): “Along similar lines, Castro Ferreira et al. (2019) propose an architecture comprising of multiple steps including discourse ordering, text structuring, lexicalization, referring expression generation, and surface realization.”
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, background): “• Data to Text: The main goal of data-to-text generation is to convert non-linguistic structured data (e.g., a table or a graph) into natural language text, which can be applied in tasks such as weather forecasting, health care [11], and so on.”
AggGen: Ordering and Aggregating while Generating (2021, background): “Castro Ferreira et al. (2019) feature a pipeline with multiple planning stages and Elder et al. (2019) introduce a symbolic intermediate representation in multi-stage neural generation.”
Improving Text-to-Text Pre-trained Models for the Graph-to-Text Task (2020, background): “Although end-to-end models trained from scratch did not outperform pipeline approaches (Nayak et al., 2017; Moryossef et al., 2019; Ferreira et al., 2019), end-to-end models pre-trained on large-scale text corpora have been shown to achieve state-of-the-art when ﬁne-tuned on the KG-to-text task…”
Controlling hallucinations at word level in data-to-text generation (2021, unknown): “Early approaches to DTG relied on static rules hand-crafted by experts, in which content selection (what to say) and surface realization (how to say it) are typically two separate tasks (Reiter and Dale 1997; Ferreira et al. 2019).”
Large Scale Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training (2020, background): “Many systems (van der Lee et al., 2018; Castro Ferreira et al., 2019; Shimorina and Gardent, 2018) have been developed for these tasks, and evaluated on these benchmark datasets, such as CRFs (Lebret et al., 2016), latent templates for interpretability (Wiseman et al., 2018), graph transformers over”
Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation (2020, methodology): “, 2019b): a DFS-based method with a neural controller; • GRU & Transformer (Ferreira et al., 2019): two neural Seq2Seq methods with attention;”
Best practices for the human evaluation of automatically generated text (2019, background): “However, we are starting to see a swing back towards a full pipeline approach with separate neural modules handling sub-tasks (Castro Ferreira et al., 2019), which may also cause a resurgence of extrinsic evaluation.”
Non-neural Models Matter: a Re-evaluation of Neural Referring Expression Generation Systems (2022, background): “Castro Ferreira et al. (2019) recently extended WEBNLG to include unseen domains that contain many unseen entities3, and Cunha et al. (2020) have developed new models to handle them.”
Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training (2021, background): “(2019) and Chen et al. (2020) also created an aligned corpus using Wikipedia hyperlinks and coreference resolution.”
Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference (2020, background): “State-of-the-art neural D2T models are prone to omitting or halluci-nating facts (Gehrmann et al., 2018; Castro Ferreira et al., 2019; Dušek et al., 2020), which restricts their real-world deployment.”
Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity (2020, methodology): “In recent years, there has been growing interest in going beyond pipeline-based approaches towards end-to-end (E-to-E) methods driven by recent advancements in deep learning (Lebret et al., 2016; Novikova et al., 2017b; Castro Ferreira et al., 2019; Duˇsek et al., 2020).”
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View (2021, methodology): “Some prior work (Moryossef, Goldberg, and Dagan 2019; Ferreira et al. 2019) have shown that the two-stage plan-based generation could lead to a more accuracy and controllable generation in datato-text generation.”
Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021, methodology): “, 2018), Transformer (Ferreira et al., 2019), Step-by-Step (Moryossef et al.”