When Exploration Comes for Free with Mixture-Greedy: Do we need UCB in Diversity-Aware Multi-Armed Bandits?

• URL: http://arxiv.org/abs/2603.21716v1
• Date: Mon, 23 Mar 2026 08:59:45 GMT
• Title: When Exploration Comes for Free with Mixture-Greedy: Do we need UCB in Diversity-Aware Multi-Armed Bandits?
• Abstract summary: We show that a simple emphMixture-Greedy strategy without explicit UCB-type optimism converges faster and achieves even better performance.<n>Under transparent structural conditions, diversity-aware objectives induce implicit exploration by favoring interior mixtures.<n>These results suggest that in diversity-aware multi-armed bandits for generative model selection, exploration can arise intrinsically from the objective geometry.
• Score: 18.528635656824864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient selection among multiple generative models is increasingly important in modern generative AI, where sampling from suboptimal models is costly. This problem can be formulated as a multi-armed bandit task. Under diversity-aware evaluation metrics, a non-degenerate mixture of generators can outperform any individual model, distinguishing this setting from classical best-arm identification. Prior approaches therefore incorporate an Upper Confidence Bound (UCB) exploration bonus into the mixture objective. However, across multiple datasets and evaluation metrics, we observe that the UCB term consistently slows convergence and often reduces sample efficiency. In contrast, a simple \emph{Mixture-Greedy} strategy without explicit UCB-type optimism converges faster and achieves even better performance, particularly for widely used metrics such as FID and Vendi where tight confidence bounds are difficult to construct. We provide theoretical insight explaining this behavior: under transparent structural conditions, diversity-aware objectives induce implicit exploration by favoring interior mixtures, leading to linear sampling of all arms and sublinear regret guarantees for entropy-based, kernel-based, and FID-type objectives. These results suggest that in diversity-aware multi-armed bandits for generative model selection, exploration can arise intrinsically from the objective geometry, questioning the necessity of explicit confidence bonuses.

Related papers

• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Mixture-Model Preference Learning for Many-Objective Bayesian Optimization [21.451177846149417]
  We propose a Bayesian framework that learns a small set of latent preference archetypes.<n>We design hybrid queries that target information about mode identity and within-mode trade-offs.<n>Our method outperforms standard baselines on synthetic and real-world many-objective benchmarks.
  arXiv  Detail & Related papers  (2026-03-30T13:18:43Z)
• FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition [22.18313011042103]
  textbfFusionAgent is a novel agentic framework for dynamic, sample-specific model selection.<n> Confidence Top-k (ACT) score-fusion anchors on the most confident model and integrates complementary predictions in a confidence-aware manner.<n>Experiments on multiple whole-body biometric benchmarks demonstrate that FusionAgent significantly outperforms SoTA methods.
  arXiv  Detail & Related papers  (2026-03-27T18:35:44Z)
• DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models [18.198091804236782]
  We introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) as a contextual bandit algorithm for the online selection of generative models with diversity considerations.<n>We show that DAK-UCB promotes diversity-aware model selection while maintaining fidelity in the generations for a sequence of prompts.
  arXiv  Detail & Related papers  (2026-03-24T12:38:26Z)
• CAMO: A Conditional Neural Solver for the Multi-objective Multiple Traveling Salesman Problem [13.232337043603161]
  We present CAMO, a conditional neural solver for the Multi-Objective Multiple Traveling Salesman Problem (MOMTSP)<n>It generalizes across varying numbers of targets, agents, and preference vectors, and yields high-quality approximations to the Pareto front (PF)<n>Experiments show that CAMO outperforms both neural and conventionals, achieving a closer approximation of PFs.
  arXiv  Detail & Related papers  (2026-03-19T15:59:45Z)
• BROTHER: Behavioral Recognition Optimized Through Heterogeneous Ensemble Regularization for Ambivalence and Hesitancy [39.407739937584104]
  Ambivalence and Hesitancy (A/H) in naturalistic video settings remains a significant challenge in affective computing.<n>We propose a highly regularized, multimodal fusion pipeline to predict A/H at the video level.
  arXiv  Detail & Related papers  (2026-03-15T12:53:41Z)
• GARDO: Reinforcing Diffusion Models without Reward Hacking [54.841464430913476]
  Fine-tuning diffusion models via online reinforcement learning (RL) has shown great potential for enhancing text-to-image alignment.<n>The mismatch often leads to reward hacking, where proxy scores increase while real image quality deteriorates and generation diversity collapses.<n>We propose Gated and Adaptive Regularization with Diversity-aware Optimization (GARDO) to address the competing demands of sample efficiency, effective exploration, and mitigation of reward hacking.
  arXiv  Detail & Related papers  (2025-12-30T10:55:45Z)
• HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generation [72.69742127579508]
  Recent unified models integrate understanding experts (e.g., LLMs) with generative experts (e.g., diffusion models)<n>In this work, we propose HBridge, an asymmetric H-shaped architecture that enables heterogeneous experts to optimally leverage pretrained priors.<n> Extensive experiments across multiple benchmarks demonstrate the effectiveness and superior performance of HBridge.
  arXiv  Detail & Related papers  (2025-11-25T17:23:38Z)
• A Unified Contrastive-Generative Framework for Time Series Classification [7.712601563682029]
  We propose a Contrastive Generative Time series framework (CoGenT)<n>It overcomes contrastive learning's sensitivity to high intra-class similarity in temporal data while reducing generative methods' dependence on large datasets.<n>The results show consistent improvements, with up to 59.2% and 14.27% F1 gains over standalone SimCLR and MAE, respectively.
  arXiv  Detail & Related papers  (2025-08-13T03:09:14Z)
• UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification [26.770271366177603]
  We propose a robust approach named Uncertainty-Guided Graph model for multi-modal object ReID (UGG-ReID)<n>UGG-ReID is designed to mitigate noise interference and facilitate effective multi-modal fusion.<n> Experimental results show that the proposed method achieves excellent performance on all datasets.
  arXiv  Detail & Related papers  (2025-07-07T03:41:08Z)
• Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles [104.60508550106618]
  We propose DiffDiv, an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs)<n>We show that DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features.<n>We show that DPM-guided diversification is sufficient to remove dependence on shortcut cues, without a need for additional supervised signals.
  arXiv  Detail & Related papers  (2023-11-23T15:47:33Z)
• BOtied: Multi-objective Bayesian optimization with tied multivariate ranks [33.414682601242006]
  In this paper, we show a natural connection between non-dominated solutions and the extreme quantile of the joint cumulative distribution function. Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied. Our experiments on a variety of synthetic and real-world problems demonstrate that BOtied outperforms state-of-the-art MOBO acquisition functions.
  arXiv  Detail & Related papers  (2023-06-01T04:50:06Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.