Semantic Parsing to Probabilistic Programs for Situated Question Answering

By Jayant Krishnamurthy, Oyvind Tafjord, Aniruddha Kembhavi. 2016

31 citations recorded by the source corpus.

Situated question answering is the problem of answering questions about an environment such as an image or diagram. This problem requires jointly interpreting a question and an environment using background knowledge to select the correct answer. We present Parsing to Probabilistic Programs (P3), a novel situated question answering model that can use background knowledge and global features of the question/environment interpretation while retaining efficient approximate inference. Our key insight is to treat semantic parses as probabilistic programs that execute nondeterministically and whose possible executions represent environmental uncertainty. We evaluate our approach on a new, publicly-released data set of 5000 science diagram questions, outperforming several competitive classical and neural baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Question Answering & Reading Comprehension, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Question Answering & Reading Comprehension

In Code Generation & Program Synthesis

Citations in the corpus

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering (2022, neutral): “8 64 1-6 natural image diagram 8 8 FOODWEBS [24] ⇡5,000 ⇡5,00 - - 8 natural (foodweb only) image diagram 8 8 ARC [5] 7,787 0 20.”
Question Answering as Global Reasoning Over Semantic Abstractions (2018, methodology): “…Khot, Sabhar-wal, and Clark (2017) use Open IE subject-verb-object relations (Etzioni et al. 2008), Banarescu et al. (2013) use AMR annotators (Wang et al. 2015), and Krishnamurthy, Tafjord, and Kembhavi (2016) use a semantic parser (Zettlemoyer and Collins 2005) to answer a given question.”
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project (2019, background): “g, food chains (Krishnamurthy et al., 2016), overall progress has been quite limited (Choi et al.”
Learning to Solve Geometry Problems from Natural Language Demonstrations in Textbooks (2017, methodology): “Technically, our deductive solver and the approach of learning from different kinds of supervision are the same as the execution model in Krishnamurthy et al. (2016).”
StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing (2018, background): “Semantic Parsing Most existing works alleviate issues of limited parallel data through weaklysupervised learning, using the denotations of MRs as indirect supervision (Reddy et al., 2014; Krishnamurthy et al., 2016; Neelakantan et al., 2016; Pasupat and Liang, 2015; Yin et al., 2016).”