Neural Semantic Parsing with Type Constraints for Semi-Structured Tables

By Jayant Krishnamurthy, Pradeep Dasigi, Matt Gardner. 2017

230 citations recorded by the source corpus.

We present a new semantic parsing model for answering compositional questions on semi-structured Wikipedia tables. Our parser is an encoder-decoder neural network with two key technical innovations: (1) a grammar for the decoder that only generates well-typed logical forms; and (2) an entity embedding and linking module that identifies entity mentions while generalizing across tables. We also introduce a novel method for training our neural model with question-answer supervision. On the WikiTableQuestions data set, our parser achieves a state-of-the-art accuracy of 43.3% for a single model and 45.9% for a 5-model ensemble, improving on the best prior score of 38.7% set by a 15-model ensemble. These results suggest that type constraints and entity linking are valuable components to incorporate in neural semantic parsers.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

NumNet: Machine Reading Comprehension with Numerical Reasoning (2019, neutral): “For comparison, we select several public models as baselines including semantic parsing models : • Syn Dep (Dua et al., 2019), the neural semantic parsing model (KDG) (Krishnamurthy et al., 2017) with Stanford dependencies based sentence representations; • OpenIE (Dua et al., 2019), KDG with open in”
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019, methodology): “4 Following, each worker answered their own question with one of three answer types: spans of text from either question or passage, a date (which was common in history and open-domain text) and numbers, allowed only for questions which explicitly stated a speciﬁc unit of measurement (e.g., “How many”
SPoC: Search-based Pseudocode to Code (2019, background): “Partial parses can be constructed by combining smaller partial parses in a bottom-up fashion [58, 28, 27], adding the next token to the sequence [10, 22, 51, 11, 25], or creating a new node in a tree structure [10, 26, 54, 39].”
A Discrete Hard EM Approach for Weakly Supervised Question Answering (2019, background): “In MML, the objective maximizes ∑ z∈Ẑ P(z|x), where Ẑ is an approximation of a set of logical forms executing y (Liang et al., 2013; Berant et al., 2013; Krishnamurthy et al., 2017).”
AllenNLP: A Deep Semantic Natural Language Processing Platform (2018, background): “…state-of-the-art for a wide range of tasks such as text classiﬁ-cation (Howard and Ruder, 2018), machine translation (Vaswani et al., 2017), semantic role labeling (Zhou and Xu, 2015; He et al., 2017), coreference resolution (Lee et al., 2017a), and semantic parsing (Krishnamurthy et al., 2017).”
Neural Module Networks for Reasoning over Text (2019, methodology): “In our model, the data types of the inputs and output of modules automatically induce a type-constrained grammar which lends itself to top-down grammar-constrained decoding as performed by Krishnamurthy et al. (2017).”
Do NLP Models Know Numbers? Probing Numeracy in Embeddings (2019, methodology): “Modeling approaches for DROP include both semantic parsing (Krishnamurthy et al., 2017) and reading comprehension (Yu et al., 2018) models.”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “Our approach to train a model to produce the latent decomposition from weak/distant supervision bears resemblance to semantic parsing approaches (Berant et al., 2013; Krishnamurthy et al., 2017) that are trained to produce the latent logical forms that can be executed over tables.”
QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions (2019, background): “, (Berant et al., 2013; Krishnamurthy et al., 2017), where the underlying qualitative comparisons are extracted and can be”
Coarse-to-Fine Decoding for Neural Semantic Parsing (2018, background): “…include tree decoders (Dong and Lapata, 2016; Alvarez-Melis and Jaakkola, 2017), decoders constrained by a grammar model (Xiao et al., 2016; Yin and Neu-big, 2017; Krishnamurthy et al., 2017), or modular decoders which use syntax to dynamically compose various submodels (Rabinovich et al., 2017).”
Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing (2019, background): “Sequence-toAction approaches (Yin and Neubig, 2017; Krishnamurthy et al., 2017; Iyyer et al., 2017; Chen et al., 2018) treat semantic parsing as the prediction of a action sequence that can construct logical forms, which not only guarantee the grammatical correctness of outputs, but also leverage th”
Question Directed Graph Attention Network for Numerical Reasoning over Text (2020, background): “All these models are enhanced versions of KDG (Krishnamurthy et al., 2017) with different sentence representations.”
Did the Model Understand the Question? (2018, background): “(2018) note that the state-of-the-art strongly supervised6 model on WikiTableQuestions (Krishnamurthy et al., 2017) enjoys a 7% gain in its final accuracy by leveraging perturbed tables during training.”
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships (2018, methodology): “This approach follows the entity-linking approach used by Krishnamurthy, Dasigi, and Gardner (2017), where the similarity between question tokens and (words associated with) entities - called the entity linking score - help decide which entities to include in the LF during parsing.”
Natural Language to Structured Query Generation via Meta-Learning (2018, background): “…studied in natu-ral language processing research (Zettlemoyer and Collins, 2005; Giordani and Moschitti, 2010; Artzi and Zettlemoyer, 2011; Berant et al., 2013; Vla-chos and Clark, 2014; Yih et al., 2014, 2015; Wang et al., 2015; Golub and He, 2016; Iyer et al., 2017; Krishnamurthy et al., 2017).”
Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing (2018, methodology): “Following [34, 24], we also add a binary feature in each step of the encoder, indicating whether this word is found in the table, and an integer feature for a column name counting how many of the words in the column name appear in the question.”
Discourse Representation Structure Parsing (2018, background): “Dong and Lapata (2016) and Alvarez-Melis and Jaakkola (2017) generate trees in a top-down fashion, while in other work (Xiao et al., 2016; Krishnamurthy et al., 2017) the decoder generates from a grammar that guarantees that predicted logical forms are well-typed.”
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing (2020, methodology): “Autoregressive top-down decoding The pre-vailing method for decoding in semantic parsing has been grammar-based autoregressive top-down decoding (Yin and Neubig, 2017; Krishnamurthy et al., 2017; Rabinovich et al., 2017), which guarantees decoding of syntactically valid programs.”
Learning to Generalize from Sparse and Underspecified Rewards (2019, background): “Previous work (Pasupat & Liang, 2016b; Mudrakarta et al., 2018; Krishnamurthy et al., 2017) has shown that efficient exploration of the search space and pruning the spurious programs”