Question analysis: How Watson reads a clue

By Adam Lally, John M. Prager, Michael C. McCord, Branimir Boguraev, Siddharth Patwardhan, James Fan, Paul Fodor, Jennifer Chu-Carroll. 2012

170 citations recorded by the source corpus.

The first stage of processing in the IBM Watson™ system is to perform a detailed analysis of the question in order to determine what it is asking for and how best to approach answering it. Question analysis uses Watson's parsing and semantic analysis capabilities: a deep Slot Grammar parser, a named entity recognizer, a co-reference resolution component, and a relation extraction component. We apply numerous detection rules and classifiers using features from this analysis to detect critical elements of the question, including: 1) the part of the question that is a reference to the answer (the focus); 2) terms in the question that indicate what type of entity is being asked for (lexical answer types); 3) a classification of the question into one or more of several broad types; and 4) elements of the question that play particular roles that may require special handling, for example, nested subquestions that must be separately answered. We describe how these elements are detected and evaluate the impact of accurate detection on our end-to-end question-answering system accuracy.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

A Neural Network for Factoid Question Answering over Paragraphs (2014, methodology): “A more apt comparison would be to IBM’s proprietary Watson system (Lally et al., 2012) for Jeopardy, which is limited to single sentences, or to models trained on Yago (Hoffart et al.”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “Traditionally, this idea has been used to compose outputs of lower-level language tasks to solve the higher-level NLU tasks (Harabagiu and Hickl, 2006; Lally et al., 2012; Khot et al., 2017; Khashabi et al., 2018b).”
YodaQA: A Modular Question Answering System Pipeline (2015, methodology): “During question analysis, produce a part-of-speech tagging and dependency parse of the question text, recognize named entities and, roughly inspired by the DeepQA system [16], heuristically generate several QA features: Clues, Focus, and LAT.”
Modeling of the Question Answering Task in the YodaQA System (2015, methodology): “For informational retrieval, we use Apache Solr 7 and include the document and title-in-clue strategies described in DeepQA [5].”
A Joint Model for Answer Sentence Ranking and Answer Extraction (2016, background): “The rule-based system is loosely inspired by the work of Lally et al. (2012), from which it differs radically because the questions in the Jeopardy! game are expressed as answers.”
A question-entailment approach to question answering (2019, methodology): “A variety of methods were developed for question analysis, focus (topic) recognition and question type identification [29, 5, 34, 33].”
Open Domain Question Answering via Semantic Enrichment (2015, methodology): “There have been many works studying the expected answer types of a question [1, 27, 28, 33, 34, 35].”