An Epistemic Position-Based Click Model: From Interactions to Epistemic Distributions of Relevance and Bias

• URL: http://arxiv.org/abs/2607.18712v1
• Date: Tue, 21 Jul 2026 05:07:50 GMT
• Title: An Epistemic Position-Based Click Model: From Interactions to Epistemic Distributions of Relevance and Bias
• Abstract summary: We introduce the first evidential deep-learning approach to form an epistemic alternative to the important position-based click model.<n>Our approach captures uncertainty about click probabilities and the underlying effects of attraction and position bias.<n>Our experiments indicate that our approach captures uncertainty in predictions on previously-unseen data, whereas standard policy gradients fail to learn meaningful distributions.
• Score: 13.021261472078605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User interactions with rankings are affected by both items' relevances and display positions. Accordingly, click probabilities are often modeled as a product of relevance and position factors; and for improving recommendation and search, one needs to disentangle relevance from position bias. However, existing click models only provide frequentist point-estimates that do not capture any measure of epistemic uncertainty. Consequently, there is no indication of how much confidence one should have in their predictions. In this work, we introduce the first evidential deep-learning approach to form an epistemic alternative to the important position-based click model. Our learned model takes as input item and position features and outputs a beta-distribution for every relevance and position-bias variable of the position-based model. These distributions capture epistemic uncertainty about click probabilities and the underlying effects of attraction and position bias. The main challenge of our approach is its optimization for which we propose approximation and conditioning techniques to provide numerical stability and variance reduction. Our experiments indicate that our approach captures epistemic uncertainty in predictions on previously-unseen data, whereas standard policy gradients fail to learn meaningful distributions. We believe our contribution of the first contextual epistemic click model constitutes an important step in incorporating Bayesian uncertainty into click modeling.

Related papers

• Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias [4.124127724629818]
  We study how a prediction model will perform in a new environment before deployment.<n>We present a double machine learning procedure for estimating the target risk of an arbitrary black-box prediction model.<n>We evaluate our estimator through experiments using the eICU electronic health records database.
  arXiv  Detail & Related papers  (2026-06-12T14:39:03Z)
• Beyond the Mean: Modelling Annotation Distributions in Continuous Affect Prediction [3.3187704612685267]
  We propose a distribution-aware framework that models annotation consensus using the Beta distribution.<n>Instead of predicting a single affect value, models estimate the mean and standard deviation of the annotation distribution.<n>As a result, the model captures not only the central tendency of emotional perception but also variability, asymmetry, and uncertainty in annotator responses.
  arXiv  Detail & Related papers  (2026-04-08T15:26:56Z)
• Frequentist Validity of Epistemic Uncertainty Estimators [9.204324119167692]
  We introduce a frequentist measure of uncertainty based on the bootstrap.<n>Our main theoretical contribution is a novel expansion that reveals that our proposed measure and the (Bayesian) mutual information are theally equivalent.<n>This provides frequentist interpretations to mutual information and new computational strategies for approximating it.
  arXiv  Detail & Related papers  (2025-10-24T22:58:42Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• An Offline Metric for the Debiasedness of Click Models [52.25681483524383]
  Click models are a common method for extracting information from user clicks.<n>Recent work shows that the current evaluation practices in the community fail to guarantee that a well-performing click model generalizes well to downstream tasks.<n>We introduce the concept of debiasedness in click modeling and derive a metric for measuring it.
  arXiv  Detail & Related papers  (2023-04-19T10:59:34Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• An Epistemic and Aleatoric Decomposition of Arbitrariness to Constrain the Set of Good Models [7.620967781722717]
  Recent research reveals that machine learning (ML) models are highly sensitive to minor changes in their training procedure.<n>We show that stability decomposes into epistemic and aleatoric components, capturing the consistency and confidence in prediction.<n>We propose a model selection procedure that includes epistemic and aleatoric criteria alongside existing accuracy and fairness criteria, and show that it successfully narrows down a large set of good models.
  arXiv  Detail & Related papers  (2023-02-09T09:35:36Z)
• Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation [51.809741427975105]
  This work aims to assess how well a model performs under distribution shifts without using labels. We use the nuclear norm that has been shown to be effective in characterizing both properties. We show that the nuclear norm is more accurate and robust in accuracy than existing methods.
  arXiv  Detail & Related papers  (2023-02-02T13:30:48Z)
• Cross Pairwise Ranking for Unbiased Item Recommendation [57.71258289870123]
  We develop a new learning paradigm named Cross Pairwise Ranking (CPR) CPR achieves unbiased recommendation without knowing the exposure mechanism. We prove in theory that this way offsets the influence of user/item propensity on the learning.
  arXiv  Detail & Related papers  (2022-04-26T09:20:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.