Query dependent pseudo-relevance feedback based on wikipedia

By Yang Xu, Gareth J. F. Jones, Bin Wang. 2009

243 citations recorded by the source corpus.

Pseudo-relevance feedback (PRF) via query-expansion has been proven to be e®ective in many information retrieval (IR) tasks. In most existing work, the top-ranked documents from an initial search are assumed to be relevant and used for PRF. One problem with this approach is that one or more of the top retrieved documents may be non-relevant, which can introduce noise into the feedback process. Besides, existing methods generally do not take into account the significantly different types of queries that are often entered into an IR system. Intuitively, Wikipedia can be seen as a large, manually edited document collection which could be exploited to improve document retrieval effectiveness within PRF. It is not obvious how we might best utilize information from Wikipedia in PRF, and to date, the potential of Wikipedia for this task has been largely unexplored. In our work, we present a systematic exploration of the utilization of Wikipedia in PRF for query dependent expansion. Specifically, we classify TREC topics into three categories based on Wikipedia: 1) entity queries, 2) ambiguous queries, and 3) broader queries. We propose and study the effectiveness of three methods for expansion term selection, each modeling the Wikipedia based pseudo-relevance information from a different perspective. We incorporate the expansion terms into the original query and use language modeling IR to evaluate these methods. Experiments on four TREC test collections, including the large web collection GOV2, show that retrieval performance of each type of query can be improved. In addition, we demonstrate that the proposed method out-performs the baseline relevance model in terms of precision and robustness.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Word-Entity Duet Representations for Document Ranking (2017, methodology): “…well-written entity descriptions and can be used as an external and cleaner pseudo relevance feedback corpus to obtain better expansion terms [24]. descriptions of related Freebase entities have been utilized to provide better expansion terms; the related entities are retrieved by entity…”
Generalizing Translation Models in the Probabilistic Relevance Framework (2016, methodology): “That is, we can either use general knowledge about the terms, extracted from exter-nal resources such as logs [5,9], manual or automatic knowledge-bases [16, 35, 37], or we can use some form of pseudo-relevance feedback (PRF) [1,18,31].”
Entity based Q&A Retrieval (2012, background): “There is also some work on using Wikipedia in general web search (Xu et al., 2009).”
Query Expansion Techniques for Information Retrieval: a Survey (2017, unknown): “29 and many articles [279,142,58,99, 149,290] have used it for probabilistic query reweighting.”
EsdRank: Connecting Query and Documents through External Semi-Structured Data (2015, methodology): “Xu et al. use pseudo relevance feedback in the Wikipedia corpus to obtain expansion terms that can be used to search other corpora [29].”
Entity query feature expansion using knowledge base links (2014, methodology): “Witheld for review mon source of external query expansion [12] [3] [36].”
Latent entity space: a novel retrieval approach for entity-bearing queries (2015, background): “Xu et al. (2009) mapped queries to Wikipedia entity pages to represent the underlying knowledge of the query and expanded the queries with the information from the mapped Wikipedia articles.”
Document Retrieval Using Entity-Based Language Models (2016, methodology): “(Only for WikiRM, the number of documents in L is selected from { 1 , 5 , 10 , 30 , 50 , 100 } following [46].)”
Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding (2017, methodology): “For example, Wikipedia articles have been used as a better corpus for pseudo relevance feedback to generate better expansion terms [28].”
Query Expansion with Freebase (2015, methodology): “Given their thorough exploration of Freebase information and use of supervision, it seems that expansion using Freebase is a rather complicated and challenging task, in which existing techniques can only provide moderate improvements.”