Structural Neural Encoders for AMR-to-text Generation

By Marco Damonte, Shay B. Cohen. 2019

73 citations recorded by the source corpus.

AMR-to-text generation is a problem recently introduced to the NLP community, in which the goal is to generate sentences from Abstract Meaning Representation (AMR) graphs. Sequence-to-sequence models can be used to this end by converting the AMR graphs to strings. Approaching the problem while working directly with graphs requires the use of graph-to-sequence models that encode the AMR graph into a vector representation. Such encoding has been shown to be beneficial in the past, and unlike sequential encoding, it allows us to explicitly capture reentrant structures in the AMR graphs. We investigate the extent to which reentrancies (nodes with multiple parents) have an impact on AMR-to-text generation by comparing graph encoders to tree encoders, where reentrancies are not preserved. We show that improvements in the treatment of reentrancies and long-range dependencies contribute to higher overall scores for graph encoders. Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of the art by 1.24 points.

Subfields: Artificial Intelligence, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Generative Models

Citations in the corpus

Tailor: Generating and Perturbing Text with Semantic Controls (2021, background): “…Ab-stract Meaning Representation (Banarescu et al., 2013; Mager et al., 2020) is an alternative semantic representation worth exploring for data perturbation, as it may further enable controls on entity recursions (Damonte and Cohen, 2019), though expressing such relationships is nontrivial.”
DART: Open-Domain Structured Data Record to Text Generation (2020, neutral): “Data-to-text datasets take different formats, including slot-value pairs, Abstract Meaning Representation (AMR) (Flanigan et al., 2016; Ferreira et al., 2017; Song et al., 2017; Zhu et al., 2019a; Ribeiro et al., 2019; Damonte and Cohen, 2019; Wang et al., 2020a; Zhao et al., 2020b; Shen et al., 202”
GPT-too: A Language-Model-First Approach for AMR-to-Text Generation (2020, background): “Damonte and Cohen (2019) compare multiple representations and ﬁnd graph encoders to be the best.”
Enhancing AMR-to-Text Generation with Dual Graph Representations (2019, unknown): “We compare our models against several state-of-the-art results reported on the two datasets (Kon-stas et al., 2017; Song et al., 2018; Beck et al., 2018; Damonte and Cohen, 2019; Cao and Clark, 2019; Guo et al., 2019).”
Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity (2020, background): “In the LDC2017T10 dataset (Knight et al., 2017), D is an Abstract Meaning Representation (AMR) graph representing “who is doing what to whom” for each sentence in T .”
A Human Evaluation of AMR-to-English Generation Systems (2020, background): “Several recent papers have explored variations on a graph-to-sequence approach: improvements in encoding reentrancies and long-range dependencies (Damonte and Cohen, 2019), a dual graph encoder that captures both top-down and bottom-up representations of graph structure (Ribeiro et al., 2019), and…”
Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation (2020, background): “However, Damonte and Cohen (2019) use both types of encoders and show that GCN can help LSTM capture reentrant structures and long-range dependencies, albeit on a different problem than ours.”
Line Graph Enhanced AMR-to-Text Generation with Mix-Order Graph Attention Networks (2020, methodology): “Such structures are identiﬁed as very difﬁcult aspects in AMR graph (Damonte and Cohen, 2019).”
AMR-To-Text Generation with Graph Transformer (2020, methodology): “4 points and the scores of t-GCNSEQ and g-GCNSEQ are improved by only 0 .”
Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs (2020, unknown): “Damonte and Cohen (2019) and Ribeiro et al. (2019) develop models employing GNNs and LSTMs, in order to learn complementary node contexts.”
Logic-Consistency Text Generation from Semantic Parses (2021, unknown): “…networks to conduct the generation on various tasks, for example, generating natural language from AMR (Song et al., 2018; Ribeiro et al., 2019; Damonte and Cohen, 2019), logic forms (Chen et al., 2020b), as well as SQL parses (Xu et al., 2018; Ngonga Ngomo et al., 2013; Koutrika et al.,…”
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward (2020, background): “Both Gated Graph Neural Networks (GGNNs) (Beck et al., 2018) and Graph Convolutional Networks (GCNs) (Damonte and Cohen, 2019) (Paulus et al., 2018; Chen and Bansal, 2018; Sharma et al., 2019).”
Structural Information Preserving for Graph-to-Text Generation (2020, background): “…2017; Song et al., 2018; Beck et al., 2018; Trisedya et al., 2018; Marcheggiani and Perez-Beltrachini, 2018; Xu et al., 2018; Cao and Clark, 2019; Damonte and Cohen, 2019; Ha-jdik et al., 2019; Koncel-Kedziorski et al., 2019; Hong et al., 2019; Song et al., 2019; Su et al., 2017) mainly studied…”
Investigating Pretrained Language Models for Graph-to-Text Generation (2020, background): “Recent approaches (Marcheggiani and Perez Bel-trachini, 2018; Song et al., 2018; Beck et al., 2018; Damonte and Cohen, 2019; Ribeiro et al., 2019; Zhao et al., 2020a; Schmitt et al., 2021; Ribeiro et al., 2021b) propose architectures based on GNNs to directly encode the graph structure, whereas…”