Refining Recency Search Results with User Click Feedback
By Taesup Moon, Wei Chu, Lihong Li, Zhaohui Zheng, Yi Chang. 2011
27 citations recorded by the source corpus.
Traditional machine-learned ranking systems for web search are often trained to capture stationary relevance of documents to queries, which has limited ability to track non-stationary user intention in a timely manner. In recency search, for instance, the relevance of documents to a query on breaking news often changes significantly over time, requiring effective adaptation to user intention. In this paper, we focus on recency search and study a number of algorithms to improve ranking results by leveraging user click feedback. Our contributions are three-fold. First, we use real search sessions collected in a random exploration bucket for \emph{reliable} offline evaluation of these algorithms, which provides an unbiased comparison across algorithms without online bucket tests. Second, we propose a re-ranking approach to improve search results for recency queries using user clicks. Third, our empirical comparison of a dozen algorithms on real-life search data suggests importance of a few algorithmic choices in these applications, including generalization across different query-document pairs, specialization to popular queries, and real-time adaptation of user clicks.
Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Katja Hofmann
- Filip Radlinski
- Ankur Gupta
- Shunbao Chen
- Jim Kleban
- Lihong Li
- Thorsten Joachims
- Sergey Levine
In Information Systems
- Katja Hofmann
- Lihong Li
- Filip Radlinski
- Ankur Gupta
- Shunbao Chen
- Jim Kleban
- Thorsten Joachims
- Maarten de Rijke
In Management Science and Operations Research
- Katja Hofmann
- Filip Radlinski
- Ankur Gupta
- Shunbao Chen
- Jim Kleban
- Thorsten Joachims
- Lihong Li
- Adith Swaminathan
Citations in the corpus
- Counterfactual Estimation and Optimization of Click Metrics for Search Engines (2014, methodology): “Furthermore, although offline evaluation was applied to recency search in the past [23], to the best of our knowledge, this work is the first to demonstrate effectiveness of counterfactual analytic techniques for Web search, including head-to-head comparisons and offline optimization in a commercial”
- Online Evaluation for Information Retrieval (2016, background): “Position bias has also been directly demonstrated in controlled experiments where a substantial difference in click-through rate is observed even when documents on the result pages are randomly shuffled [Moon et al., 2012].”