Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search

By Thorsten Joachims, Laura A. Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, Geri Gay. 2007

700 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

Citations in the corpus

A Framework for Data-Driven Robotics (2019, background): “Learning from preferences [43]–[48] also has a long history.”
Click Prediction for Web Image Reranking Using Multimodal Sparse Coding (2014, background): “Joachims et al. [39] conducted an eye-tracking experiment to observe the relationship between the clicked links and the relevance of the target pages, while Shokouhi et al. [8] investigated the effect of reordering web search results based on click through search effectiveness.”
Online Evaluation for Information Retrieval (2016, background): “CTR has been shown to be noisy and strongly biased, particularly due to document position [Joachims et al., 2007].”
Understanding web browsing behaviors through Weibull analysis of dwell time (2010, background): “, [12, 13, 24]), and the other based on large-scale log analysis (e.”
No clicks, no problem: using cursor movements to understand and improve search (2011, methodology): “Comparing Click and Hover Distributions Prior studies have presented data on click distribution [18,24] or gaze distribution for the search results [5,18].”
Evaluation of contextual information retrieval effectiveness: overview of issues and research (2010, background): “It refers to the user’s levels of expertise [73] and user interests either short-term ones [1,43,57,91] or long-term ones [70,93,103].”
Reducing Dueling Bandits to Cardinal Bandits (2014, background): “ing to a classic body of work in econometrics and empirical work in information retrieval (Joachims et al., 2007), such implicit feedback is typically viewed as an ordinal preference between alternatives (i.”
Detecting duplicate web documents using clickthrough data (2011, result): “results independently of document relevance) makes inferring relevance from usage data difficult [21, 25, 16], our results show that different types of presentation bias occur when different types of duplicates occur in search results.”
Beliefs and biases in web search (2013, background): “Joachims et al. [20] analyzed searchers’ decision processes via gaze tracking and compared implicit feedback from search-result clicks against manual relevance judgments.”
Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “For example, recent studies on eye tracking demonstrate that users are less likely to browse items that are ranked lower in vertical lists, while they only examine the first few items at the top of lists [34], [35].”
Tailoring click models to user goals (2009, background): “Previous eyetracking studies [9, 10] indicated that clicks are biased as a form of absolute relevance judgement, and clicking decision on a web document depends on both the position (rank) and the context (other documents) of the presentation.”
Scaling data-driven robotics with reward sketching and batch reinforcement learning (2019, background): “Learning from preferences has a long history [66, 50, 19, 64, 14, 32].”
Online Structured Prediction via Coactive Learning (2012, methodology): “In particular, we focus on preference feedback from clicks in web-search and draw upon data from a user study (Joachims et al., 2007).”
MergeDTS (2018, background): “Since user feedback is known to be noisy [17, 28, 29, 38], how to infer ranker quality and when to stop evaluating a ranker are two important challenges in online ranker evaluation.”
A user browsing model to predict search engine click data from past observations. (2008, methodology): “Joachims [12] among others sustains this view.”