Eliminating Search Intent Bias in Learning to Rank

• URL: http://arxiv.org/abs/2002.03203v2
• Date: Tue, 11 Feb 2020 23:11:58 GMT
• Title: Eliminating Search Intent Bias in Learning to Rank
• Authors: Yingcheng Sun and Richard Kolacinski and Kenneth Loparo
• Abstract summary: We study how differences in user search intent can influence click activities and determined that there exists a bias between user search intent and the relevance of the document relevance. We propose a search intent bias hypothesis that can be applied to most existing click models to improve their ability to learn unbiased relevance.
• Score: 0.32228025627337864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Click-through data has proven to be a valuable resource for improving search-ranking quality. Search engines can easily collect click data, but biases introduced in the data can make it difficult to use the data effectively. In order to measure the effects of biases, many click models have been proposed in the literature. However, none of the models can explain the observation that users with different search intent (e.g., informational, navigational, etc.) have different click behaviors. In this paper, we study how differences in user search intent can influence click activities and determined that there exists a bias between user search intent and the relevance of the document relevance. Based on this observation, we propose a search intent bias hypothesis that can be applied to most existing click models to improve their ability to learn unbiased relevance. Experimental results demonstrate that after adopting the search intent hypothesis, click models can better interpret user clicks and substantially improve retrieval performance.

Related papers

• Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Dataset [48.708591046906896]
  Unbiased learning-to-rank (ULTR) is a well-established framework for learning from user clicks. We revisit and extend the available experiments on the Baidu-ULTR dataset. We find that standard unbiased learning-to-rank techniques robustly improve click predictions but struggle to consistently improve ranking performance.
  arXiv  Detail & Related papers  (2024-04-03T08:00:46Z)
• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases [62.54519787811138]
  We present a simple but effective method to measure and mitigate model biases caused by reliance on spurious cues. We rank images within their classes based on spuriosity, proxied via deep neural features of an interpretable network. Our results suggest that model bias due to spurious feature reliance is influenced far more by what the model is trained on than how it is trained.
  arXiv  Detail & Related papers  (2022-12-05T23:15:43Z)
• Whole Page Unbiased Learning to Rank [59.52040055543542]
  Unbiased Learning to Rank(ULTR) algorithms are proposed to learn an unbiased ranking model with biased click data. We propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model. Experimental results on a real-world dataset verify the effectiveness of the BAL.
  arXiv  Detail & Related papers  (2022-10-19T16:53:08Z)
• Measurement and applications of position bias in a marketplace search engine [0.0]
  Search engines intentionally influence user behavior by picking and ranking the list of results. This paper describes our efforts at Thumbtack to understand the impact of ranking. We include a novel discussion of how ranking bias may not only affect labels, but also model features.
  arXiv  Detail & Related papers  (2022-06-23T14:09:58Z)
• A Graph-Enhanced Click Model for Web Search [67.27218481132185]
  We propose a novel graph-enhanced click model (GraphCM) for web search. We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
  arXiv  Detail & Related papers  (2022-06-17T08:32:43Z)
• Scalar is Not Enough: Vectorization-based Unbiased Learning to Rank [29.934700345584726]
  Unbiased learning to rank aims to train an unbiased ranking model from biased user click logs. Most of the current ULTR methods are based on the examination hypothesis (EH), which assumes that the click probability can be factorized into two scalar functions. We propose a vector-based EH and formulate the click probability as a dot product of two vector functions.
  arXiv  Detail & Related papers  (2022-06-03T17:23:25Z)
• Hidden Biases in Unreliable News Detection Datasets [60.71991809782698]
  We show that selection bias during data collection leads to undesired artifacts in the datasets. We observed a significant drop (>10%) in accuracy for all models tested in a clean split with no train/test source overlap. We suggest future dataset creation include a simple model as a difficulty/bias probe and future model development use a clean non-overlapping site and date split.
  arXiv  Detail & Related papers  (2021-04-20T17:16:41Z)
• Query Abandonment Prediction with Recurrent Neural Models of Mouse Cursor Movements [10.088906689243768]
  We show that mouse cursor movements make a valuable, low-cost behavioral signal that can discriminate good and bad abandonment. Our results can help search providers to gauge user satisfaction for queries without clicks and ultimately contribute to a better understanding of search engine performance.
  arXiv  Detail & Related papers  (2021-01-22T11:57:04Z)
• Handling Position Bias for Unbiased Learning to Rank in Hotels Search [0.951828574518325]
  We will investigate the importance of properly handling the position bias in an online test environment in Tripadvisor Hotels search. We propose an empirically effective method of handling the position bias that fully leverages the user action data. The online A/B test results show that this method leads to an improved search ranking model.
  arXiv  Detail & Related papers  (2020-02-28T03:48:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.