Correcting Selection Bias in Sparse User Feedback for Large Language Model Quality Estimation: A Multi-Agent Hierarchical Bayesian Approach

• URL: http://arxiv.org/abs/2605.12177v1
• Date: Tue, 12 May 2026 14:22:06 GMT
• Title: Correcting Selection Bias in Sparse User Feedback for Large Language Model Quality Estimation: A Multi-Agent Hierarchical Bayesian Approach
• Abstract summary: We propose a three-agent hierarchical Bayesian pipeline that does not require ground-truth labels on individual interactions.<n>A mild prior on the feedback channel (typical positive-feedback rate and negative-to-positive ratio) keeps Hierarchical-Informed within 4-13 pp of $Qstar$ as the bias ratio sweeps.<n>Without channel-side priors, every weak-prior variant misses $Qstar$ by 22-33 pp.
• Score: 0.9558392439655014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: [Abridged] Production LLM deployments receive feedback from a non-random fraction of users: thumbs sit mostly in the tails of the satisfaction distribution, and a naive average over them can land 40-50 percentage points away from true system quality. We treat this as a topic- and sentiment- stratified selection-bias problem and propose a three-agent hierarchical Bayesian pipeline that does not require ground-truth labels on individual interactions. A Topic Clustering Agent partitions the stream via UMAP + HDBSCAN over text embeddings; a Bias Modeling Agent fits a two-stage hierarchical Beta-Binomial under NUTS, inferring per-topic selection rates $s_c$ and quality $q_c$ with partial pooling; a Synthesis Agent reweights $q_c$ by true topic prevalence $\hatπ_c = n_c/N$ to report a bias-corrected aggregate posterior $\bar Q = \sum_c \hatπ_c q_c$ with credible interval, plus drift signals for online recalibration. Validation uses UltraFeedback (N=10,232 retained interactions, $C=18$ clusters, $Q^\star=0.6249$) with simulated topic- and sentiment-dependent selection biases. We compare five Bayesian variants against Naive and IPW baselines. A mild prior on the feedback channel (typical positive-feedback rate and negative-to-positive ratio, both readable from any production dashboard without labels) keeps Hierarchical-Informed within 4-13 pp of $Q^\star$ as the bias ratio sweeps from 1:1 to 30:1, with 95% credible intervals covering $Q^\star$ in 50/50 random-seed replicates at $κ_{\max}=10$. Without channel-side priors, every weak-prior variant misses $Q^\star$ by 22-33 pp: the per-cluster sufficient statistics admit a one-parameter family of equally good fits, and the prior on the bias channel (not on latent quality) is what breaks the degeneracy.

Related papers

• Conditioning Tree-Based Diffusions and Flows for Probabilistic Tabular Regression [0.0]
  Tree-based diffusion models fit flexible conditional predictive distributions for regression without a neural density estimator.<n>We show these defaults are the binding constraint: what a gradient-boosted ensemble actually solves is a supervised regression problem.<n>We present DiffGBM, which makes them explicit along two axes.
  arXiv  Detail & Related papers  (2026-07-30T22:08:02Z)
• Interface-Variant Dynamics in Software Ecosystems: Resolver-Induced Selection and Adoption in Package Graphs [0.2864713389096699]
  This paper asks when that observation is a load-bearing software-engineering estimator rather than evolutionary relabeling.<n>We mine interface histories, audit npm, Central, PyPI, and crates.io package graphs, execute 2100 package-manager resolver probes, estimate an ecosystem-specific selection $s$ from clean conflict probabilities, and use that measured $s$ to forward evaluate a pairwise-comparison absorbing process on the observed package graph.
  arXiv  Detail & Related papers  (2026-06-30T15:29:39Z)
• BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence [22.315546054051143]
  Bias audits of large language models now operate within governance frameworks such as the EU AI Act.<n>We introduce textscBiAxisAudit, a protocol that reports each bias score together with a reliability estimate on two axes.
  arXiv  Detail & Related papers  (2026-05-09T16:26:49Z)
• Measuring the Unmeasurable: Markov Chain Reliability for LLM Agents [0.7161783472741748]
  We present textscTraceToChain, a reproducible pipeline that fits agent execution traces to a Markov chain.<n>The pipeline builds an automatic cluster taxonomy, estimates transitions with Laplace-smoothed maximum-likelihood estimation (MLE), and checks fit with a composite Akaike information criterion (AIC) and Kolmogorov--Smirnov (KS) goodness-of-fit certificate.<n>The resulting first-passage view reconciles metrics usually reported separately: pass$@k$, pass$k$, and the reliability decay curve (RDC) are projections of one
  arXiv  Detail & Related papers  (2026-04-27T15:05:45Z)
• Reinforcement Learning from Multi-Source Imperfect Preferences: Best-of-Both-Regimes Regret [71.69884486156359]
  We study episodic RL from emphmulti-source imperfect preferences through a cumulative imperfection budget.<n>We propose a unified algorithm with regret $tildeO(sqrtK/M+)$, which exhibits a best-of-both-regimes behavior.
  arXiv  Detail & Related papers  (2026-03-20T19:34:53Z)
• When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines [0.0]
  Multi-agent LLM pipelines produce contradictory evidence on whether team diversity improves output quality.<n>We propose a resolution by identifying the selection bottleneck that determines whether diversity helps or hurts.<n>Our results suggest that selector quality may be a more impactful design lever than generator diversity in single-round generate-then-select pipelines.
  arXiv  Detail & Related papers  (2026-03-20T00:50:53Z)
• Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models [13.46898179372249]
  Vision-Language Models (VLMs) have become indispensable for multimodal reasoning, yet their representations often encode and amplify demographic biases.<n>We propose a geometrically principled framework that identifies and removes the entire subspace of linearly decodable bias.<n>Our method achieves more robust debiasing with an average improvement of $18.5%$ across four fairness metrics.
  arXiv  Detail & Related papers  (2025-11-22T17:04:30Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Regression-Based Estimation of Causal Effects in the Presence of Selection Bias and Confounding [52.1068936424622]
  We consider the problem of estimating the expected causal effect $E[Y|do(X)]$ for a target variable $Y$ when treatment $X$ is set by intervention.<n>In settings without selection bias or confounding, $E[Y|do(X)] = E[Y|X]$, which can be estimated using standard regression methods.<n>We propose a framework that incorporates both selection bias and confounding.
  arXiv  Detail & Related papers  (2025-03-26T13:43:37Z)
• Transfer Q Star: Principled Decoding for LLM Alignment [105.89114186982972]
  Transfer $Q*$ estimates the optimal value function for a target reward $r$ through a baseline model. Our approach significantly reduces the sub-optimality gap observed in prior SoTA methods.
  arXiv  Detail & Related papers  (2024-05-30T21:36:12Z)
• Bias Mimicking: A Simple Sampling Approach for Bias Mitigation [57.17709477668213]
  We introduce a new class-conditioned sampling method: Bias Mimicking. Bias Mimicking improves underrepresented groups' accuracy of sampling methods by 3% over four benchmarks.
  arXiv  Detail & Related papers  (2022-09-30T17:33:00Z)
• Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees [43.13918072870693]
  We propose an optimistic posterior sampling algorithm for reinforcement learning (OPSRL) We guarantee a high-probability regret bound of order at most $widetildemathcalO(sqrtH3SAT)$ ignoring $textpolylog(HSAT)$ terms. Our bound matches the lower bound of order $Omega(sqrtH3SAT)$, thereby answering the open problems raised by Agrawal and Jia.
  arXiv  Detail & Related papers  (2022-09-28T20:49:34Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.