Creating Training Corpora for NLG Micro-Planners

By Claire Gardent, Anastasia Shimorina, Laura Perez-Beltrachini. 2017

492 citations recorded by the source corpus.

In this paper, we present a novel framework for semi-automatically creating linguistically challenging micro-planning data-to-text corpora from existing Knowledge Bases. Because our method pairs data of varying size and shape with texts ranging from simple clauses to short texts, a dataset created using this framework provides a challenging benchmark for microplanning. Another feature of this framework is that it can be applied to any large scale knowledge base and can therefore be used to train and learn KB verbalisers. We apply our framework to DBpedia data and compare the resulting dataset with Wen et al. 2016’s. We show that while Wen et al.’s dataset is more than twice larger than ours, it is less diverse both in terms of input and in terms of text. We thus propose our corpus generation framework as a novel method for creating challenging data sets from which NLG models can be learned which are capable of handling the complex interactions occurring during in micro-planning between lexicalisation, aggregation, surface realisation, referring expression generation and sentence segmentation. To encourage researchers to take up this challenge, we made available a dataset of 21,855 data/text pairs created using this framework in the context of the WebNLG shared task.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “[cz] • WebNLG 2020 (Gardent et al., 2017; Castro Ferreira et al., 2020) – Verbalize subject-predicate-object triples in one or more sentences in a grammatical and natural way.”
Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task (2021, methodology): “In this paper, we apply these techniques to 3 of the datasets from the GEM Shared Task (Gehrmann et al., 2021): the Schema-Guided Dialog (SGD) dataset (Rastogi et al., 2019), the End-to-End (E2E) dataset (Novikova et al., 2017) and the WebNLG-en dataset (Gardent et al., 2017).”
Survey of Hallucination in Natural Language Generation (2022, methodology): “Data-to-Text Generation is the task of generating natural language descriptions conditioned on structured data, such as tables [100, 149], and knowledge graphs [41].”
DART: Open-Domain Structured Data Record to Text Generation (2020, methodology): “WebNLG 2017 The WebNLG dataset (Gardent et al., 2017a) is a set of triples extracted from DB-pedia and the human-annotated target text.”
The WebNLG Challenge: Generating Text from RDF Data (2017, methodology): “In (Gardent et al., 2017), we compared a dataset created using the WebNLG process with existing benchmarks in particular, (Wen et al., 2016)’s dataset ( RNNLG ) which was produced using a similar process.”
ToTTo: A Controlled Table-To-Text Generation Dataset (2020, background): “Target Quality Target Source Content Selection Wikibio (Lebret et al., 2016) 583K Biographies Noisy Wikipedia Not speciﬁed Rotowire (Wiseman et al., 2017) 4.9K Basketball Noisy Rotowire Not speciﬁed WebNLG (Gardent et al., 2017b) 25.3K 15 DBPedia categories Clean Annotator Generated Fully speciﬁed E”
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (2019, methodology): “We use the NYT (Riedel et al., 2010) and WebNLG (Gardent et al., 2017) datasets to evaluate the proposed method.”
Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism (2018, unknown): “The second is WebNLG dataset (Gardent et al., 2017).”
Controlling hallucinations at word level in data-to-text generation (2021, unknown): “In particular, surface-realization abilities have been well studied on hand-crafted datasets such as E2E [43] and WebNLG [14], while content-selection has been addressed by automatically constructed dataset such as WikiBio [29] or RotoWire [70].”
RDF2PT: Generating Brazilian Portuguese Texts from RDF Data (2018, methodology): “In the questionnaire, we used the same two questions as Gardent et al. (2017): (1) Adequacy: Does the text contain only and all the information from the data?”
GTR-LSTM: A Triple Encoder for Sentence Generation from RDF Data (2018, methodology): “The ﬁrst is the dataset from Gardent et al. (2017a).”
Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers (2020, unknown): “To address this issue, Gardent et al. (2017a) came up with a new dataset which includes RDFs extracted from DBPedia 1 with their corresponding translations in natural language.”
The 2020 Bilingual, Bi-Directional WebNLG+ Shared Task: Overview and Evaluation Results (WebNLG+ 2020) (2020, unknown): “procedure as was followed for the collection of the initial WebNLG data (Gardent et al., 2017a), but without the verification step.”
InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation (2021, methodology): “…grammatically, conciseness, and readability (Mani 2001; Colombo et al. 2022), whereas machine translation focuses on ﬁdelity, ﬂuency and adequacy of the translation (Hovy 1999) and data2text generation (Gardent et al. 2017) consider criteria such as data coverage, correctness and text structure.”
Describing a Knowledge Base (2018, methodology): “Our task is similar to the WebNLG challenge generating text from DBPedia data (Gardent et al., 2017a).”
Deep Graph Convolutional Encoders for Structured Data to Text Generation (2018, methodology): “The WebNLG task (Gardent et al., 2017a,b) aims at the generation of entity descriptions from a set of RDF triples related to an entity of a given category (Perez-Beltrachini et al., 2016).”