GPT-too: A Language-Model-First Approach for AMR-to-Text Generation

By Manuel Mager, Ramón Fernandez Astudillo, Tahira Naseem, Md Arafat Sultan, Young-suk Lee, Radu Florian, Salim Roukos. 2020

106 citations recorded by the source corpus.

Abstract Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequence-to-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10 dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Tailor: Generating and Perturbing Text with Semantic Controls (2021, background): “Ab-stract Ab-stract Meaning Representation (Banarescu et al., 2013; Mager et al., 2020) is an alternative semantic representation worth exploring for data perturbation, as it may further enable controls on entity recursions (Damonte and Cohen, 2019), though expressing such relationships is…”
GraDA: Graph Generative Data Augmentation for Commonsense Reasoning (2022, background): “Additionally, the graph-to-text generative models in G RA DA pose the same risk as other data-to-text generative models (Ribeiro et al., 2020; Hoyle et al., 2020; Mager et al., 2020) i.e. the models can be made to generate incorrect facts by providing incorrect data as input.”
A Human Evaluation of AMR-to-English Generation Systems (2020, background): “, 2019), generating syntactic constituency trees as an intermediate step before generating surface structure (Cao and Clark, 2019), and fine-tuning GPT-2 on AMR (Mager et al., 2020).”
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations (2022, unknown): “Similar to AMR parsing, we use ﬁnetuned T5 3 that is shown to be effective for the AMR-to-Text generation task (Mager et al., 2020; Ribeiro et al., 2020).”
Structural Information Preserving for Graph-to-Text Generation (2020, methodology): “GPT-2L (Mager et al., 2020) applies a GPT-2 Large model (Radford et al.”
Investigating Pretrained Language Models for Graph-to-Text Generation (2020, methodology): “Following previous work (Kon-stas et al., 2017; Mager et al., 2020 In this experiment we employ T5 small .”
Pre-Trained Language Models for Text Generation: A Survey (2022, methodology): “Data-to-text Generation AGENDA [158], LDC2017T10 [130], WikiBio [29], WebNLG [158], E2E [25] BLEU, ROUGE, METEOR, chrF++”
GraDA: Graph Generative Data Augmentation for Commonsense Reasoning (2022, background): “Several recent works have shown that pretrained generative models can be finetuned with or without structure-aware graph encoding to improve graph-to-text generation (Mager et al., 2020; Ribeiro et al., 2020; Hoyle et al., 2020; He et al., 2020; Ke et al., 2021).”