Building Watson: An Overview of the DeepQA Project

By David A. Ferrucci, E. Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John M. Prager and 2 more. 2010

1,590 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016, background): “The annual evaluations at the Text REtreival Conference (TREC) (Voorhees and Tice, 2000) led to many advances in open-domain QA, many of which were used in IBM Watson for Jeopardy! (Ferrucci et al., 2013).”
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020, methodology): “…al., 2019), a semi-automatic Italian translation of SQuAD (Croce et al., 2018), ARCD—an Arabic reading comprehension dataset (Mozannar et al., 2019), a Hindi-English parallel dataset in a SQuAD-like setting (Gupta et al., 2018), and a Chinese–English dataset focused on visual QA (Gao et al., 2015).”
WebGPT: Browser-assisted question-answering with human feedback (2021, methodology): “One notable system of this kind was DeepQA (also known as IBM Watson), which was used to beat the best humans at Jeopardy [Ferrucci et al., 2010].”
Reading Wikipedia to Answer Open-Domain Questions (2017, methodology): “, 2002), IBM’s DeepQA (Ferrucci et al., 2010) and YodaQA (Baudiš, 2015; Baudiš and Šedivỳ, 2015) – the latter of which is open source and hence reproducible for comparison purposes.”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “Studies in the early stage (Kwok et al., 2001; Brill et al., 2002; Ferrucci et al., 2010; Baudiˇs, 2015) mostly rely on highly sophisticated pipelines and heterogeneous resources.”
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “Text-based open-domain QA is a long studied problem (Voorhees et al., 1999; Ferrucci et al., 2010).”
Learning Dense Representations of Phrases at Scale (2020, background): “Open-domain question answering (QA) aims to provide answers to natural-language questions using a large text corpus (Voorhees et al., 1999; Ferrucci et al., 2010; Chen and Yih, 2020).”
Unsupervised Question Decomposition for Question Answering (2020, methodology): “IBM Watson (Ferrucci et al., 2010) decomposes questions into sub-questions in multiple ways or not at all.”
Achieving Human Parity on Visual Question Answering (2021, methodology): “Traditional approaches typically use a pipeline method by considering question processing, document retrieval and answer reranking such as the famous IBM Watson system [19].”
Visual7W: Grounded Question Answering in Images (2015, background): “Traditional question answering system relies on an elaborate pipeline of models involving natural language parsing, knowledge base querying, and answer generation [6].”
A Deep Reinforcement Learning Chatbot (2017, methodology): “Examples of such ensemble systems include the winner of the Netﬂix Prize (Koren et al. 2009), utilizing hundreds of machine learning models to predict user movie preferences, and IBM Watson (Ferrucci et al. 2010), the ﬁrst machine learning system to win the quiz game Jeopardy! in 2011.”
Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions (2016, methodology): “Answering questions using an ensemble has been shown to be effective in numerous previous cases, (e.g., T¨oscher, Jahrer, and Bell (2009)), most famously in IBM’s highly successful Watson system (Ferrucci et al. 2010).”
Batch Policy Gradient Methods for Improving Neural Conversation Models (2017, methodology): “Besides chatbots, this framework also finds applications in tasks such as question answering (Ferrucci et al., 2010; Hermann et al., 2015; Sachan et al., 2016), generating image descriptions (Karpathy & Fei-Fei, 2015) and machine translation (Bahdanau et al.”