Beyond Static Bias: Adaptive Multi-Fidelity Bandits with Improving Proxies

• URL: http://arxiv.org/abs/2605.08558v1
• Date: Fri, 08 May 2026 23:36:53 GMT
• Title: Beyond Static Bias: Adaptive Multi-Fidelity Bandits with Improving Proxies
• Abstract summary: Multi-fidelity multiarmed bandits (MF-MAB) enable individual arms to be evaluated using diverse feedback sources that vary in both cost and accuracy.<n>We investigate adaptive MF-MAB with improving proxy sources, and focus on the canonical two-fidelity case in which the low-fidelity source becomes more informative with repeated use.<n>We propose the Threshold-Based Adaptive Continuation Companion (TACC), an optimistic algorithm that uses a bounded continuation rule to decide when low-fidelity sampling remains cost-effective and when to escalate.
• Score: 16.078266766067838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As an extension of the classical multi-armed bandit problem, multi-fidelity multi-armed bandits (MF-MAB) enable individual arms to be evaluated using diverse feedback sources that vary in both cost and accuracy. Prior stochastic models typically assume fixed low-to-high fidelity discrepancies, whereas modern proxy sources, such as learning-based simulators and Large Language Models (LLMs), can be improved using additional calibration. We investigate adaptive MF-MAB with improving proxy sources, and focus on the canonical two-fidelity case in which the low-fidelity source becomes more informative with repeated use. To capture this dynamic, we introduce a selected-average mismatch bound that converts dynamic low-fidelity observations into improvement-aware confidence bounds for the high-fidelity target. We propose the Threshold-Based Adaptive Continuation Companion (TACC), an optimistic algorithm that uses a bounded continuation rule to decide when low-fidelity sampling remains cost-effective and when to escalate. We prove an instance-dependent regret bound showing that, for detected intermediate arms, adaptive continuation replaces logarithmic high-fidelity confirmation with bounded low-fidelity continuation. Experiments on synthetic bandits and an LLM-as-a-judge policy-evaluation task examine when continuation improves cost-weighted regret.

Related papers

• Progressive Content Refinement with Decaying Reward Joint LinUCB [2.1290878226779877]
  We propose a novel contextual bandit algorithm that explicitly incorporates reward decay modeling.<n>We show that our method achieves significant performance gains over strong baselines.<n>Our ablation study confirms that the integration of reward decay modeling within the bandit framework is crucial for mitigating over-exploitation.
  arXiv  Detail & Related papers  (2026-08-07T03:17:22Z)
• Adaptive Confidence-weighted Expansion for Trustworthy Multi-Omics Multimodal Fusion [0.8749675983608171]
  We introduce Adaptive Confidence-weighted Expansion (ACE) to enhance the trustworthiness of multimodal fusion models.<n>ACE significantly outperforms existing state-of-the-art algorithms in both classification performance and confidence calibration.
  arXiv  Detail & Related papers  (2026-07-22T21:42:16Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Architecture-agnostic Lipschitz-constant Bayesian header and its application to resolve semantically proximal classification errors with vision transformers [0.0]
  This work presents an architecture-agnostic Lipschitz-constant Bayesian header that can be integrated into feature extractors such as vision transformers.<n>We also propose a novel metric to capture uncertainty and confidence across misclassification rates, as well as an adaptive arithmetic-mean fusion scheme.<n>Although computational costs increase due to Monte Carlo sampling, the method offers plug-and-play compatibility with pre-trained backbones.
  arXiv  Detail & Related papers  (2026-05-07T09:18:06Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Less Approximates More: Harmonizing Performance and Confidence Faithfulness via Hybrid Post-Training for High-Stakes Tasks [52.81286869496811]
  Large language models are increasingly deployed in high-stakes tasks, where confident yet incorrect inferences may cause real-world harm.<n>We propose to jointly optimize unsupervised Reinforcement Learning from Internal Feedback with reasoning-trace-guided Reasoning Distillation.<n>We introduce HyTuning, a hybrid post-training framework that adaptively reweights RD and RLIF via a PRG-style metric.
  arXiv  Detail & Related papers  (2026-04-09T16:50:11Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Fairness Begins with State: Purifying Latent Preferences for Hierarchical Reinforcement Learning in Interactive Recommendation [23.667292152163068]
  We present textbfDSRM-HRL, a framework that reformulates fairness-aware recommendation as a latent state purification problem.<n>We show that DSRM-HRL effectively breaks the "rich-get-richer" feedback loop, achieving a superior frontier between recommendation utility and exposure equity.
  arXiv  Detail & Related papers  (2026-03-04T08:14:21Z)
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning [88.42566960813438]
  CalibRL is a hybrid-policy RLVR framework that supports controllable exploration with expert guidance.<n>CalibRL increases policy entropy in a guided manner and clarifies the target distribution.<n>Experiments across eight benchmarks, including both in-domain and out-of-domain settings, demonstrate consistent improvements.
  arXiv  Detail & Related papers  (2026-02-22T07:23:36Z)
• Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling [49.41422138354821]
  We propose a principled reward modeling framework that integrates non-negative factor analysis into the Bradley-Terry preference model.<n>BNRM represents rewards through a sparse, non-negative latent factor generative process.<n>We show that BNRM substantially mitigates reward over-optimization, improves robustness under distribution shifts, and yields more interpretable reward decompositions than strong baselines.
  arXiv  Detail & Related papers  (2026-02-11T08:14:11Z)
• FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models [3.8824066002669855]
  Multi-fidelity (MF) regression often operates in regimes of extreme data imbalance.<n>We introduce FIRE, a training-free MF framework.<n>Fire delivers a stronger performance-time trade-off than seven state-of-the-art GP-based or deep learning MF regression methods.
  arXiv  Detail & Related papers  (2026-01-29T22:29:58Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• A Novel Generative Model with Causality Constraint for Mitigating Biases in Recommender Systems [20.672668625179526]
  Latent confounding bias can obscure the true causal relationship between user feedback and item exposure.<n>We propose a novel generative framework called Latent Causality Constraints for Debiasing representation learning in Recommender Systems.
  arXiv  Detail & Related papers  (2025-05-22T14:09:39Z)
• A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling [18.02518660778453]
  adaptive sampling methods that dynamically allocate resources among fidelity models can achieve higher efficiency in the exploring and exploiting the design space. We propose a framework hinged on a latent embedding for different fidelity models and the associated pre-posterior analysis to explicitly utilize their correlation for adaptive sampling. We demonstrate that the proposed method outperforms the benchmark methods in both MF global fitting (GF) and Bayesian Optimization (BO) problems in convergence rate and robustness.
  arXiv  Detail & Related papers  (2023-10-05T03:56:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.