Query performance prediction in web search environments

By Yun Zhou, W. Bruce Croft. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2007

310 citations recorded by the source corpus.

Current prediction techniques, which are generally designed for content-based queries and are typically evaluated on relatively homogenous test collections of small sizes, face serious challenges in web search environments where collections are significantly more heterogeneous and different types of retrieval tasks exist. In this paper, we present three techniques to address these challenges. We focus on performance prediction for two types of queries in web search environments: content-based and Named-Page finding. Our evaluation is mainly performed on the GOV2 collection. In addition to evaluating our models for the two types of queries separately, we consider a more challenging and realistic situation that the two types of queries are mixed together without prior information on query types. To assist prediction under the mixed-query situation, a novel query classifier is adopted. Results show that our prediction of web query performance is substantially more accurate than the current state-of-the-art prediction techniques. Consequently, our paper provides a practical approach to performance prediction in real-world web settings.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/56f944d0fb742d23ad8abcd1727c3871b577a021

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Latent entity space: a novel retrieval approach for entity-bearing queries (2015, methodology): “As the optimal k value is correlated with query difﬁculty, existing query performance prediction methods (He and Ounis 2006; Zhou and Croft 2007) could be leveraged to better predict k based on performance prediction results and further improve the performance.”
A comparison of user and system query performance predictions (2010, methodology): “Automatically predicting the performance (or retrieval effectiveness), of queries is a very active area of research [8, 9, 14, 36, 38, 39, 40].”
Adding semantics to microblog posts (2012, background): “WIG ( q ) indicates the weighted information gain [39], which can be considered a predictor of the retrieval performance of a query.”
To personalize or not to personalize: modeling queries with variation in user intent (2008, methodology): “[26] Zhou, Y. and Croft, W. B. (2007).”
Discovering key concepts in verbose queries (2008, background): “In our paper, a normalized version of WIG is used 1 P Md log p(ci|d) - log p(ci|C) TM (ci) wig(ci)= , - log p(ci|C) where TM (ci) is a set of top M documents3 retrieved in response to ci from collection C, and p(ci|·) is calculated using a maximum likelihood estimate [25].”