Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

By Ioannis Konstas, Srini Iyer, Mark Yatskar, Yejin Choi, Luke S. Zettlemoyer. 2017

317 citations recorded by the source corpus.

Sequence-to-sequence models have shown strong performance across a broad range of applications. However, their application to parsing and generating text using Abstract Meaning Representation (AMR) has been limited, due to the relatively limited amount of labeled data and the non-sequential nature of the AMR graphs. We present a novel training procedure that can lift this limitation using millions of unlabeled sentences and careful preprocessing of the AMR graphs. For AMR parsing, our model achieves competitive results of 62.1 SMATCH, the current best score reported without significant use of external semantic resources. For AMR generation, our model establishes a new state-of-the-art performance of BLEU 33.8. We present extensive ablative and qualitative analysis including strong evidence that sequence-based AMR models are robust against ordering variations of graph-to-sequence conversions.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Autoregressive Entity Retrieval (2020, background): “, 2019), generation and parsing of abstract meaning representation (Konstas et al., 2017).”
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech (2018, methodology): “n the input or target sequences, and has been successfully applied to tasks such as machine translation [21, 22], video caption generation [23], abstract meaning representation parsing and generation [24], and acoustic word embeddings acquisition [11]. arXiv:1803.08976v2 [cs.CL] 9 Jun 2018 (a) Speec”
Neural Motifs: Scene Graph Parsing with Global Context (2017, background): “Indeed, graph linearization has worked surprisingly well for many problems in vision and language, such as generating image captions from object detections [52], language parsing [44], generating text from abstract meaning graphs [21].”
Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue (2019, background): “Also related are neural generators that take Abstract Meaning Representations (AMRs) as input (Konstas et al., 2017, i.a.) rather than flat inputs; these approaches, however, do not generate trees or use constrained decoding.”
End-to-End Content and Plan Selection for Data-to-Text Generation (2018, methodology): “To address this issue, Konstas et al. (2017) propose a semi-supervised training method that can utilize English sentences outside of the training set to train parts of the model.”
A Sequence-to-Sequence Model for Semantic Role Labeling (2018, methodology): “More closely related to SRL is the AMR pars- ing and generation system proposed by Konstas et al. (2017).”
Self-Training for Jointly Learning to Ask and Answer Questions (2018, methodology): “Most prominently, they have been used for word sense disambiguation (Yarowsky, 1995), noun learning (Riloff et al., 2003) and recently, AMR parsing and generation (Konstas et al., 2017).”
Learning an Executable Neural Semantic Parser (2017, methodology): “…in various ways, such as multi-task learning (Fan et al. 2017), parsing cross-domain queries (Herzig and Berant 2017) and context-dependent queries (Suhr, Iyer, and Artzi 2018), and applying the model to other formalisms such as AMR (Konstas et al. 2017) and SQL (Zhong, Xiong, and Socher 2017).”
Graph-to-Sequence Learning using Gated Graph Neural Networks (2018, methodology): “We use the list obtained from the open source implementation of (Konstas et al., 2017) for that.”
Structural Information Preserving for Graph-to-Text Generation (2020, methodology): “…group corresponds to the reported numbers of previous models on this dataset, and their main difference is the encoder for presenting graphs: LSTM (Konstas et al., 2017) applies a multi-layer LSTM on linearized AMRs, GRN (Song et al., 2018) and DCGCN (Guo et al., 2019) adopt graph neural…”
Structural Neural Encoders for AMR-to-text Generation (2019, methodology): “The sequential encoder (S EQ ) is a re-implementation of Konstas et al. (2017).”
Linguistic realisation as machine translation: Comparing different MT models for AMR-to-text generation (2017, methodology): “On the other hand, similar to Konstas et al. (2017), we delexicalised these constants, replacing the original information for tags (e.g., name1 , quant1 ).”
Deep Graph Convolutional Encoders for Structured Data to Text Generation (2018, methodology): “For the SR11Deep base-line we follow a similar linearisation procedure as proposed for AMR graphs (Konstas et al., 2017).”
Text Generation from Knowledge Graphs with Graph Transformers (2019, background): “Konstas et al. (2017) provide the ﬁrst neural model for this task, and show that pretraining on a large dataset of noisy automatic parses can improve results.”
Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (2018, methodology): “We cast this task to a S EQ 2S EQ transduction task by linearizing the AMR graphs (Konstas et al., 2017).”
Broad-Coverage Semantic Parsing as Transduction (2019, background): “, 2017b), but has relied on data augmentation to produce effective parsers (van Noord and Bos, 2017; Konstas et al., 2017).”
A Human Evaluation of AMR-to-English Generation Systems (2020, methodology): “Shortly after the 2017 shared task, Konstas et al. (2017) made signiﬁcant advances to the ﬁeld with a neural sequence-to-sequence approach, mitigating the limitations of the small amount of AMR-annotated data by augmenting training data with a jointly-trained parser.”
Learning Word Embeddings from Speech (2017, methodology): “…of challenging tasks such as machine translation [Sutskever et al., 2014, Cho et al., 2014], video caption generation [Venugopalan et al., 2015], abstract meaning representation (AMR) parsing and generation [Konstas et al., 2017], and acquisition of acoustic word embeddings [Chung et al., 2016].”