Generating Text from Structured Data with Application to the Biography Domain

By Rémi Lebret, David Grangier, Michael Auli. 2016

51 citations recorded by the source corpus.

This paper introduces a neural model for concept-to-text generation that scales to large, rich domains. We experiment with a new dataset of biographies from Wikipedia that is an order of magni- tude larger than existing resources with over 700k samples. The dataset is also vastly more diverse with a 400k vocab- ulary, compared to a few hundred words for Weathergov or Robocup. Our model builds upon recent work on conditional neural language model for text genera- tion. To deal with the large vocabulary, we extend these models to mix a fixed vocabulary with copy actions that trans- fer sample-specific words from the in- put database to the generated output sen- tence. Our neural model significantly out- performs a classical Kneser-Ney language model adapted to this task by nearly 15 BLEU.

Subfields: Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Generative Models

Citations in the corpus

Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth (2021, methodology): “To compute these ratios we ran the network on 32 samples of 128 tokens excerpts of biographies from Wikipedia [LGA16] and display the mean and standard deviation.”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, methodology): “There has also been some work applying nns to data-to-text generation (Mei et al., 2016; Lebret et al., 2016).”
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023, methodology): “…hallu-cination detection approaches by 1) generating synthetic Wikipedia articles using GPT-3 on the individuals/concepts from the WikiBio dataset (Lebret et al., 2016); 2) manually annotating the factuality of the passage at a sentence level; 3) evaluating the system’s ability to detect…”
Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples (2017, methodology): “stances of number-objects are replaced with the special token 0, except for year-objects that are mapped to the special <year> token [10].”
Poisoning Web-Scale Training Datasets is Practical (2023, background): “, the WikiQA [81] question answering dataset (30,000+ downloads), and the WikiBio [38] biography writing dataset (19,000+ downloads).”
Table-to-text Generation by Structure-aware Seq2seq Learning (2017, methodology): “Lebret, Grangier, and Auli (2016) proposed a statistical n-gram model with local and global conditioning on a Wikipedia infobox.”
Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data (2019, methodology): “For this task, we use the WIKIBIO dataset (Lebret et al., 2016) as the benchmark dataset.”