Deep Questions without Deep Understanding

By Igor Labutov, Sumit Basu, Lucy Vanderwende. 2015

135 citations recorded by the source corpus.

We develop an approach for generating deep (i.e, high-level) comprehension questions from novel text that bypasses the myriad challenges of creating a full semantic representation. We do this by decomposing the task into an ontologycrowd-relevance workflow, consisting of first representing the original text in a low-dimensional ontology, then crowdsourcing candidate question templates aligned with that space, and finally ranking potentially relevant templates for a novel region of text. If ontological labels are not available, we infer them from the text. We demonstrate the effectiveness of this method on a corpus of articles from Wikipedia alongside human judgments, and find that we can generate relevant deep questions with a precision of over 85% while maintaining a recall of 70%.

Subfields: Artificial Intelligence, Question Answering & Reading Comprehension, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Question Answering & Reading Comprehension

In Natural Language Generation

Citations in the corpus

Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering (2019, neutral): “Question Generation Early QG studies focused on using rule-based methods to transform statements to questions (Heilman and Smith, 2010; Lindberg et al., 2013; Labutov et al., 2015).”
Towards a Better Metric for Evaluating Question Generation Systems (2018, background): “…systems (ii) studies which analyze evaluation metrics used for NLG. Question Generation: Early work on question generation used rule-based approaches to generate questions from declarative sentences (Heilman and Smith, 2010; Mostow and Chen, 2009; Lind-berg et al., 2013; Labutov et al., 2015).”
Learning to Ask: Neural Question Generation for Reading Comprehension (2017, background): “Labutov et al. (2015) use crowdsourcing to collect a set of templates and then rank the relevant templates for the text of another domain.”
Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks (2018, background): “One is built on top of heuristic rules that creates questions with manually constructed template and ranks the generated results, e.g. (Heil-man and Smith, 2010; Mazidi and Nielsen, 2014; Labutov et al., 2015).”
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (2018, methodology): “Most early work in QG employed rule-based approaches to transform input text into questions, usually requiring the application of a sequence of well-designed general rules or templates (Mitkov and Ha, 2003; Labutov et al., 2015).”
Answer-based Adversarial Training for Generating Clarification Questions (2019, background): “Labutov et al. (2015) use crowdsourcing to generate question templates whereas we learn from naturally occurring questions.”
A Joint Model for Question Answering and Question Generation (2017, unknown): “Earlier work on question generation has resorted to either rule-based reordering methods (Heilman & Smith, 2010; Agarwal & Mannem, 2011; Ali et al., 2010) or slot-ﬁlling with question templates (Popowich & Winne, 2013; Chali & Golestanirad, 2016; Labutov et al., 2015).”
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (2018, methodology): “Outside reading comprehension questions, Labutov et al. (2015) generate high-level question templates by crowdsourcing which leads to significantly less data than we collect using our method.”
Question Generation for Question Answering (2017, methodology): “Labutov et al. (2015) proposed an ’ontology-crowd-relevance’ method for question generation.”
Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards (2017, background): “In [1, 4], authors focus on automatically generating gap-ﬁll questions, while crowdsourcing templates and manually built templates are used for question generation in [15] and [19] respectively.”
Neural Models for Key Phrase Extraction and Question Generation (2018, methodology): “…proposed methods resort to simple, rule-based techniques such as template-based slot ﬁll-ing (Lindberg et al., 2013; Chali and Golestanirad, 2016; Labutov et al., 2015) or syntactic transformation heuristics (Agarwal and Mannem, 2011; Ali et al., 2010) (e.g., subject-auxiliary inversion,…”
A Recurrent BERT-based Model for Question Generation (2019, background): “One is built on top of heuristic rules that creates questions with manually constructed template and ranks the generated results (Heilman and Smith, 2010; Mazidi and Nielsen, 2014; Labutov et al., 2015).”
Machine Comprehension by Text-to-Text Neural Question Generation (2017, methodology): “It has been harnessed, for example, as a means to build automatic tutoring systems (Heilman and Smith, 2010; Ali et al., 2010; Lindberg et al., 2013; Labutov et al., 2015; Mazidi and Nielsen, 2015), to reroute queries to community question-answering systems (Zhao et al., 2011), and to enrich…”
Question Answering and Question Generation as Dual Tasks (2017, background): “(Labutov, Basu, and Vanderwende 2015) propose an ontology-crowd-relevance approach to generate questions from novel text.”
Generating Natural Questions About an Image (2016, background): “…al., 2012) create ﬁll-in-the-blank questions, (Mazidi and Nielsen, 2014) and (Lindberg et al., 2013) use manually constructed question templates, while (Labutov et al., 2015) use crowd-sourcing to collect a set of templates and then rank the potentially relevant templates for the selected content.”