Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information

By Sudha Rao, Hal Daumé Iii. 2018

227 citations recorded by the source corpus.

Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clarification questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clarification questions consisting of 77K posts paired with a clarification question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate significant improvements over controlled baselines.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

AmbigQA: Answering Ambiguous Open-domain Questions (2020, background): “Research on community question answering (Braslavski et al., 2017; Rao and Daum´e III, 2018, 2019) models ambiguity that arises from common information gaps in the question, which are often speciﬁc to the community being studied.”
Learning from Dialogue after Deployment: Feed Yourself, Chatbot! (2019, background): “Learning to ask questions is another area that has been studied (Strub et al., 2017; Wang et al., 2018; Rao and III, 2018).”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “tives, and ordinary words at each decoding position. The final output distribution is modeled by the type distribution, leading to a strong control over the question to be generated.Rao and Daumé III [99] also argued that question asking is fundamental to communication, and that a good question is t”
Why Build an Assistant in Minecraft? (2019, methodology): “This has been studied in (ungrounded) conversational systems [77, 90, 68, 49] and when grounding with images via the CLEVR QA task [62].”
Asking Clarification Questions in Knowledge-Based Question Answering (2019, background): “Unlike previous clariﬁcation-related datasets with limited annotated examples (De Boni and Manandhar, 2003; Stoyanchev et al., 2014) or in speciﬁc domains (Li et al., 2017; Rao and III, 2018), our dataset covers various domains and supports three tasks.”
MIMICS: A Large-Scale Data Collection for Search Clarification (2020, background): “Clarification has been explored in a number of applications, such as speech recognition [26], dialogue systems [4, 12, 20], and community question answering [5, 23, 24].”
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020, background): “…have been studied in multiple domains to resolve ambiguity caused by speech recognition failure (Stoyanchev et al., 2014), recover missing information in question answering (Rao and Daum´e III, 2018) or clarify information needs in open-domain information-seeking (Aliannejadi et al., 2019).”
Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation (2019, background): “Asking appropriate questions in practical applications has recently been paid attention (Rao & Daum´e III, 2018; Buck et al., 2018).”
Asking Clarifying Questions in Open-Domain Information-Seeking Conversations (2019, background): “These studies rely on large amount of data from industrial chatbots [31, 46], query logs [37], and QAwebsites [34, 35, 41].”