Improving Web Search Ranking by Incorporating User Behavior Information
By Eugene Agichtein, Eric Brill, Susan T. Dumais. 2019
326 citations recorded by the source corpus.
We show that incorporating user behavior data can significantly improve ordering of top results in real web search setting. We examine alternatives for incorporating feedback into the ranking process and explore the contributions of user feedback compared to other common web search features. We report results of a large scale evaluation over 3,000 queries and 12 million user interactions with a popular web search engine. We show that incorporating implicit feedback can augment other features, improving the accuracy of a competitive web search ranking algorithms by as much as 31% relative to the original performance.
Subfields: Information Systems, Information Retrieval & Search
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Systems
- Hamed Zamani
- Min Zhang
- Michael Bendersky
- Yiqun Liu
- Shaoping Ma
- Xuanhui Wang
- Yongfeng Zhang
- W. Bruce Croft
In Information Retrieval & Search
- Kurt Shuster
- Arthur Szlam
- Mojtaba Komeili
- Jian-Yun Nie
- Jason Aaron Edward Weston
- Xuanhui Wang
- Yiqun Liu
- Michael Bendersky
Citations in the corpus
- BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “, 2010), improving web search (Agichtein et al., 2006), the Dynabench system which evaluates a number of NLP tasks (Kiela et al.”
- Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls (2022, background): “Further, a continual deployment of such a system can then potentially keep improving over time (Carlson et al., 2010; Kiela et al., 2021; Agichtein et al., 2006; Liu and Mazumder, 2021; Madotto et al., 2020; Shus-ter et al., 2020), constituting what Hancock et al. (2019) call a “self-feeding…”
- Filter Bubbles, Echo Chambers, and Online News Consumption (2016, unknown): “Additionally, search engines, news aggregators and social networks are increasingly personalizing content through machine learned algorithms (Agichtein et al., 2006; Das et al., 2007; Hannak et al., 2013), which can in principle create “filter bubbles” (Pariser, 2011) that amplify ideological segreg”
- Filter-enhanced MLP is All You Need for Sequential Recommendation (2022, unknown): “Besides, these methods mainly fit the model parameters based on logged user behavior data, which are in essence noisy [1, 43] or even contains malicious fakes [8, 65].”
- Situational Context for Ranking in Personal Search (2017, background): “It should be noted that, since we, in this paper, are interested in studying the affect of situational features, we do not consider user’s search histories, although, they have shown to be effective in web search [1, 24, 50].”
- Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback (2022, background): “…are numerous studies attempting to improve language skills from deployment, including never-ending-learning from language data (Carlson et al., 2010), learning for the web search task directly (Agichtein et al., 2006) or the Dynabench system which covers a number of NLP tasks (Kiela et al., 2021).”
- Time-Aware Click Model (2016, methodology): “Although click-through logs can provide implicit feedback of users’ click preferences [Agichtein et al. 2006b], it is difficult to derive accurate absolute relevance judgments owing to the existence of click noises and behavior biases. Joachims et al. [2005] worked on extracting reliable implicit fe”