Transfer Learning for Neural Semantic Parsing

By Xing Fan, Emilio Monti, Lambert Mathias, Markus Dreyer. 2017

65 citations recorded by the source corpus.

The goal of semantic parsing is to map natural language to a machine interpretable meaning representation language (MRL). One of the constraints that limits full exploration of deep learning technologies for semantic parsing is the lack of sufficient annotation training data. In this paper, we propose using sequence-to-sequence in a multi-task setup for semantic parsing with focus on transfer learning. We explore three multi-task architectures for sequence-to-sequence model and compare their performance with the independently trained model. Our experiments show that the multi-task setup aids transfer learning from an auxiliary task with large labeled data to the target task with smaller labeled data. We see an absolute accuracy gain ranging from 1.0% to 4.4% in in our in-house data set and we also see good gains ranging from 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and semantic auxiliary tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Coarse-to-Fine Decoding for Neural Semantic Parsing (2018, background): “, 2016; Jia and Liang, 2016), transfer learning (Fan et al., 2017), sharing parameters for multiple languages or meaning representations (Susanto and Lu, 2017; Herzig and Berant, 2017), and utilizing user feedback signals (Iyer et al.”
Learning an Executable Neural Semantic Parser (2017, methodology): “Other related work extends the vanilla sequence-to-sequence model in various ways, such as multi-task learning (Fan et al. 2017), parsing cross-domain queries (Herzig and Berant 2017) and context-dependent queries (Suhr, Iyer, and Artzi 2018), and applying the model to other formalisms such as AMR (”
Efficient Large-Scale Domain Classification with Personalized Attention (2018, methodology): “Multi-Task learning was used in the context of SLU by Tur (2006) and has been further explored using neural networks for phoneme recognition (Seltzer and Droppo, 2013) and semantic parsing (Fan et al., 2017; Bapna et al., 2017).”
Building a Neural Semantic Parser from a Domain Ontology (2018, methodology): “…ways, such as employing two encoder-decoders for coarse to ﬁne decoding (Dong & Lapata, 2018), handling multiple tasks with a shared encoder (Fan, Monti, Mathias, & Dreyer, 2017), parsing cross-domain queries (Herzig & Berant, 2017) and context-dependent queries (Suhr, Iyer, & Artzi, 2018),…”
Statistical learning for semantic parsing: A survey (2019, methodology): “In Reference [95], the authors achieved good performance of a seq-to-seq model with three multi-task architectures.”
Practical Semantic Parsing for Spoken Language Understanding (2019, methodology): “Our copy implementation follows the strategy of Fan et al. (2017), where the output of the generation layer is concatenated to the scores of an attention mechanism (Bahdanau et al., 2015), which express the relevance of each input word with respect to the current state of the parser.”
Semantic Parsing for Task Oriented Dialog using Hierarchical Representations (2018, methodology): “, 2013) and Alexa Meaning Representation (Fan et al., 2017).”