Eliminating Search Intent Bias in Learning to Rank

• URL: http://arxiv.org/abs/2002.03203v2
• Date: Tue, 11 Feb 2020 23:11:58 GMT
• Title: Eliminating Search Intent Bias in Learning to Rank
• Authors: Yingcheng Sun and Richard Kolacinski and Kenneth Loparo
• Abstract summary: We study how differences in user search intent can influence click activities and determined that there exists a bias between user search intent and the relevance of the document relevance. We propose a search intent bias hypothesis that can be applied to most existing click models to improve their ability to learn unbiased relevance.
• Score: 0.32228025627337864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Click-through data has proven to be a valuable resource for improving search-ranking quality. Search engines can easily collect click data, but biases introduced in the data can make it difficult to use the data effectively. In order to measure the effects of biases, many click models have been proposed in the literature. However, none of the models can explain the observation that users with different search intent (e.g., informational, navigational, etc.) have different click behaviors. In this paper, we study how differences in user search intent can influence click activities and determined that there exists a bias between user search intent and the relevance of the document relevance. Based on this observation, we propose a search intent bias hypothesis that can be applied to most existing click models to improve their ability to learn unbiased relevance. Experimental results demonstrate that after adopting the search intent hypothesis, click models can better interpret user clicks and substantially improve retrieval performance.

Related papers

• Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance [74.43264459255121]
  In real-world scenarios, users often click only one or two results after examining multiple relevant options. We propose a query-level click propensity model to capture the probability that users will click on different result lists. Our method introduces a Dual Inverse Propensity Weighting mechanism to address both relevance saturation and position bias.
  arXiv  Detail & Related papers  (2025-02-17T03:55:51Z)
• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• An Offline Metric for the Debiasedness of Click Models [52.25681483524383]
  Click models are a common method for extracting information from user clicks. Recent work shows that the current evaluation practices in the community fail to guarantee that a well-performing click model generalizes well to downstream tasks. We introduce the concept of debiasedness in click modeling and derive a metric for measuring it.
  arXiv  Detail & Related papers  (2023-04-19T10:59:34Z)
• Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases [62.54519787811138]
  We present a simple but effective method to measure and mitigate model biases caused by reliance on spurious cues. We rank images within their classes based on spuriosity, proxied via deep neural features of an interpretable network. Our results suggest that model bias due to spurious feature reliance is influenced far more by what the model is trained on than how it is trained.
  arXiv  Detail & Related papers  (2022-12-05T23:15:43Z)
• Whole Page Unbiased Learning to Rank [59.52040055543542]
  Unbiased Learning to Rank(ULTR) algorithms are proposed to learn an unbiased ranking model with biased click data. We propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model. Experimental results on a real-world dataset verify the effectiveness of the BAL.
  arXiv  Detail & Related papers  (2022-10-19T16:53:08Z)
• Measurement and applications of position bias in a marketplace search engine [0.0]
  Search engines intentionally influence user behavior by picking and ranking the list of results. This paper describes our efforts at Thumbtack to understand the impact of ranking. We include a novel discussion of how ranking bias may not only affect labels, but also model features.
  arXiv  Detail & Related papers  (2022-06-23T14:09:58Z)
• A Graph-Enhanced Click Model for Web Search [67.27218481132185]
  We propose a novel graph-enhanced click model (GraphCM) for web search. We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
  arXiv  Detail & Related papers  (2022-06-17T08:32:43Z)
• Handling Position Bias for Unbiased Learning to Rank in Hotels Search [0.951828574518325]
  We will investigate the importance of properly handling the position bias in an online test environment in Tripadvisor Hotels search. We propose an empirically effective method of handling the position bias that fully leverages the user action data. The online A/B test results show that this method leads to an improved search ranking model.
  arXiv  Detail & Related papers  (2020-02-28T03:48:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.