AMR-To-Text Generation with Graph Transformer

By Tian-ming Wang, Xiaojun Wan, Hanqi Jin. 2020

79 citations recorded by the source corpus.

Abstract meaning representation (AMR)-to-text generation is the challenging task of generating natural language texts from AMR graphs, where nodes represent concepts and edges denote relations. The current state-of-the-art methods use graph-to-sequence models; however, they still cannot significantly outperform the previous sequence-to-sequence models or statistical approaches. In this paper, we propose a novel graph-to-sequence model (Graph Transformer) to address this task. The model directly encodes the AMR graphs and learns the node representations. A pairwise interaction function is used for computing the semantic relations between the concepts. Moreover, attention mechanisms are used for aggregating the information from the incoming and outgoing neighbors, which help the model to capture the semantic information effectively. Our model outperforms the state-of-the-art neural approach by 1.5 BLEU points on LDC2015E86 and 4.8 BLEU points on LDC2017T10 and achieves new state-of-the-art performances.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Natural Language Generation

Citations in the corpus

Extracting Temporal Event Relation with Syntactic-Guided Temporal Graph Transformer (2021, methodology): “Our work is also related to the variants of Graph Neural Networks (GNN) (Kipf and Welling, 2016; Veliˇckovi´c et al., 2017; Zhou et al., 2018), especially Graph Transformers (Yun et al., 2019; Chen et al., 2019; Hu et al., 2020; Wang et al., 2020b).”
GPT-too: A Language-Model-First Approach for AMR-to-Text Generation (2020, methodology): “Using transformers has also been recently explored by Wang et al. (2020) who propose a mutli-head graph attention mechanism and by Cai and Lam (2020) who propose a graph transformer architecture.”
Structural Information Preserving for Graph-to-Text Generation (2020, result): “…al., 2019; Ribeiro et al., 2019) have been introduced to better represent input AMRs than a sequence-to-sequence model (Konstas et al., 2017), and later work (Zhu et al., 2019; Cai and Lam, 2019; Wang et al., 2020) showed that relation-aware Transformers can achieve even better results than GNNs.”
Investigating Pretrained Language Models for Graph-to-Text Generation (2020, background): “…et al., 2021; Ribeiro et al., 2021b) propose architectures based on GNNs to directly encode the graph structure, whereas other efforts (Ribeiro et al., 2020; Schmitt et al., 2020; Yao et al., 2020; Wang et al., 2020) inject the graph structure information into Transformer-based architectures.”