Learning surface text patterns for a Question Answering System

By Deepak Ravichandran, Eduard H. Hovy. 2002

959 citations recorded by the source corpus.

In this paper we explore the power of surface text patterns for open-domain question answering systems. In order to obtain an optimal set of patterns, we have developed a method for learning such patterns automatically. A tagged corpus is built from the Internet in a bootstrapping process by providing a few hand-crafted examples of each question type to Altavista. Patterns are then automatically extracted from the returned documents and standardized. We calculate the precision of each pattern, and the average precision for each question type. These patterns are then applied to find answers to new questions. Using the TREC-10 question set, we report results for two cases: answers determined from the TREC-10 corpus and from the web.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016, methodology): “…a question is similar to answer extraction , the ﬁnal step in the open-domain QA pipeline, methods for which include bootstrapping surface patterns (Ravichandran and Hovy, 2002), using dependency trees (Shen and Klakow, 2006), and using a factor graph over multiple sentences (Sun et al., 2013).”
How Can We Know What Language Models Know? (2019, methodology): “Our first method is inspired by templatebased relation extraction methods (Agichtein and Gravano, 2000; Ravichandran and Hovy, 2002), which are based on the observation that words in the vicinity of the subject x and object y in a large corpus often describe the relation r.”
Distant supervision for relation extraction without labeled data (2009, methodology): “A third approach has been to use a very small number of seed instances or patterns to do bootstrap learning (Brin, 1998; Riloff and Jones, 1999; Agichtein and Gravano, 2000; Ravichandran and Hovy, 2002; Etzioni et al., 2005; Pennacchiotti and Pantel, 2006; Bunescu and Mooney, 2007; Rozenfeld and Fel”
Pattern Learning for Relation Extraction with a Hierarchical Topic Model (2012, background): “Manually selected seeds can also be used (Ravichandran and Hovy, 2002; Kozareva and Hovy, 2010).”
Learning Syntactic Patterns for Automatic Hypernym Discovery (2004, background): “Thus a wide variety of recent research has focused on finding methods for automatically learning taxonomic relations and constructing semantic hierarchies [1, 2, 3, 4, 6, 8, 9, 10, 16, 18, 19, 20, 21, 22].”
Split and Rephrase (2017, background): “from the new templates (Ravichandran and Hovy, 2002; Duclaye et al., 2003); systems which extract paraphrase patterns from large monolingual corpora and use them to rewrite an input text (Duboue and Chu-Carroll, 2006; Narayan et al.”
ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters (2018, background): “Traditionally, the problem of QA has been explored over large textual corpora (Cui et al., 2005; Dietz and Gamari, 2017; Ferrucci, 2012; Harabagiu et al., 2001, 2003; Ravichandran and Hovy, 2002; Saquete et al., 2009; Voorhees and Tice, 2000) with answers being textual phrases.”
The Distributional Inclusion Hypotheses and Lexical Entailment (2005, background): “A few recent works (Ravichandran and Hovy, 2002; Keller et al., 2002; Chklovski and Pantel, 2004) used the web to collect statistics on word co-occurrences.”
Crowdsourcing Inference-Rule Evaluation (2012, background): “Inference rules are an important component in semantic applications, such as Question Answering (QA) (Ravichandran and Hovy, 2002) and Information Extraction (IE) (Shinyama and Sekine, 2006), describing a directional inference relation between two text patterns with variables.”
Unsupervised Personal Name Disambiguation (2003, methodology): “In this paper, we employ and extend the method described by Ravichandran and Hovy (2002) shown in Figure 1.”
Corpus analysis of simultaneous interpretation data for improving real time speech translation (2013, background): “Lexical compression refers to the use of fewer words to express the same idea and has been extensively studied as a paraphrasing problem [10, 11].”
Structured Relation Discovery using Generative Models (2011, background): “For instance, knowing that the entity Steve Balmer bears the leaderOf relation to the entity Microsoft , would facilitate question answering (Ravichandran and Hovy, 2002), data mining, and a host of other end-user applications.”
Unsupervised named-entity extraction from the Web: An experimental study (2005, background): “Like our work, Ravichandran and Hovy [36] use Web search engines to ﬁnd patterns surrounding seed values.”
Mining the peanut gallery: opinion extraction and semantic classification of product reviews (2003, background): “” Substitutions have been used to solve these sorts of problems in subjectivity identification, text classification, and question answering [18, 19, 26], but, as indicated in Table 3, they are mostly ineffective for our task.”
Quantifying Similarity between Relations with Fact Distribution (2019, methodology): “Hence, there are accordingly various methods proposed for discovering more relations and their facts, including open information extraction (Brin, 1998; Agichtein and Gravano, 2000; Ravichandran and Hovy, 2002; Banko et al., 2007; Zhu et al., 2009; Etzioni et al., 2011; Saha et al., 2017) and relati”
Large Scale Acquisition of Paraphrases for Learning Surface Patterns (2008, methodology): “For example, in question answering, paraphrases have been used to ﬁ nd multiple patterns that pinpoint the same answer (Ravichandran and Hovy, 2002); in statistical machine translation, they have been used to ﬁ nd translations for unseen source language phrases (Callison-Burch et al., 2006); in…”
Discovering Relations among Named Entities from Large Corpora (2004, methodology): “Ravichan-dran also explored a similar method for question answering (Ravichandran and Hovy, 2002).”