DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models

• URL: http://arxiv.org/abs/2603.23140v2
• Date: Wed, 25 Mar 2026 08:13:16 GMT
• Title: DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models
• Abstract summary: We introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) as a contextual bandit algorithm for the online selection of generative models with diversity considerations.<n>We show that DAK-UCB promotes diversity-aware model selection while maintaining fidelity in the generations for a sequence of prompts.
• Score: 18.198091804236782
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The expansion of generative AI and LLM services underscores the growing need for adaptive mechanisms to select an appropriate available model to respond to a user's prompts. Recent works have proposed offline and online learning formulations to identify the optimal generative AI model for an input prompt, based solely on maximizing prompt-based fidelity evaluation scores, e.g., CLIP-Score in text-to-image generation. However, such fidelity-based selection methods overlook the diversity of generated outputs, and hence, they can fail to address potential diversity shortcomings in the generated responses. In this paper, we introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) method as a contextual bandit algorithm for the online selection of generative models with diversity considerations. The proposed DAK-UCB method incorporates both fidelity and diversity-related metrics into the selection process. We design this framework based on prompt-aware diversity score functions that decompose to a two-sample-based expectation over prompt-output pairs in the previous generation rounds. Specifically, we illustrate the application of our framework using joint kernel distance and kernel entropy measures. Our experimental results demonstrate the effectiveness of DAK-UCB in promoting diversity-aware model selection while maintaining fidelity in the generations for a sequence of prompts. The code is available at https://github.com/Donya-Jafari/DAK-UCB.

Related papers

• When Exploration Comes for Free with Mixture-Greedy: Do we need UCB in Diversity-Aware Multi-Armed Bandits? [18.528635656824864]
  We show that a simple emphMixture-Greedy strategy without explicit UCB-type optimism converges faster and achieves even better performance.<n>Under transparent structural conditions, diversity-aware objectives induce implicit exploration by favoring interior mixtures.<n>These results suggest that in diversity-aware multi-armed bandits for generative model selection, exploration can arise intrinsically from the objective geometry.
  arXiv  Detail & Related papers  (2026-03-23T08:59:45Z)
• DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing [78.70918589095639]
  Reinforcement learning (RL)-based enhancement of large language models (LLMs) often leads to reduced output diversity.<n>This paper proposes an RL framework structured around a semi-structured long Chain-of-Thought (CoT)<n>We introduce a Diverse Planning Branching method that strategically introduces divergence at the planning phase based on diversity variation.
  arXiv  Detail & Related papers  (2026-01-14T16:30:20Z)
• UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings [70.60608084375691]
  We pioneer the exploration of generative embeddings, unifying embedding tasks within a generative paradigm.<n>We propose UME-R1, a universal multimodal embedding framework consisting of a two-stage training strategy.<n> evaluated on the MMEB-V2 benchmark across 78 tasks spanning video, image, and visual documents.
  arXiv  Detail & Related papers  (2025-11-01T05:04:23Z)
• Post-training Large Language Models for Diverse High-Quality Responses [32.92680825196664]
  Reinforcement learning (RL) has emerged as a popular method for post-training large language models (LLMs)<n>We propose a novel training method named DQO (Diversity Quality Optimization) based on determinantal point processes (DPPs)<n>Our approach samples and embeds a group of responses for each prompt, then uses the determinant of a kernel-based similarity matrix to measure diversity as the volume spanned by the embeddings of these responses.
  arXiv  Detail & Related papers  (2025-09-05T03:47:06Z)
• Jointly Reinforcing Diversity and Quality in Language Model Generations [64.72289248044514]
  Post-training of Large Language Models (LMs) often prioritizes accuracy and helpfulness at the expense of diversity.<n>We address this challenge with Diversity-Aware Reinforcement Learning (DARLING), a framework that jointly optimize for response quality and semantic diversity.
  arXiv  Detail & Related papers  (2025-09-02T17:38:47Z)
• SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score [22.364171814102487]
  Diffusion models have demonstrated remarkable success in high-fidelity image synthesis and prompt-guided generative modeling.<n>We propose the Scalable Prompt-Aware R'eny Kernel Entropy Diversity Guidance (SPARKE) method for prompt-aware diversity guidance.<n>We numerically test the SPARKE method on several text-to-image diffusion models, demonstrating that the proposed method improves the prompt-aware diversity of the generated data without incurring significant computational costs.
  arXiv  Detail & Related papers  (2025-06-11T20:53:45Z)
• MMaDA: Multimodal Large Diffusion Language Models [61.13527224215318]
  We introduce MMaDA, a novel class of multimodal diffusion foundation models.<n>It is designed to achieve superior performance across diverse domains such as textual reasoning, multimodal understanding, and text-to-image generation.
  arXiv  Detail & Related papers  (2025-05-21T17:59:05Z)
• Evaluating the Diversity and Quality of LLM Generated Content [72.84945252821908]
  We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds.<n>Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models.<n>These findings have important implications for applications that require diverse yet high-quality outputs.
  arXiv  Detail & Related papers  (2025-04-16T23:02:23Z)
• Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting [9.116108409344177]
  The source-free cross-domain few-shot learning task aims to transfer pretrained models to target domains utilizing minimal samples.<n>We propose the SeGD-VPT framework, which is divided into two phases.<n>The first step aims to increase feature diversity by adding diversity prompts to each support sample, thereby generating varying input and enhancing sample diversity.
  arXiv  Detail & Related papers  (2024-12-01T11:00:38Z)
• PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs [38.167419614117044]
  In this work, we propose an online learning framework to predict the best data generation model for a given input prompt.<n>The proposed PAK-UCB algorithm addresses a contextual bandit (CB) setting with shared context variables across the arms.<n>Our experiments on real and simulated text-to-image and image-to-text generative models show that RFF-UCB performs successfully in identifying the best generation model.
  arXiv  Detail & Related papers  (2024-10-17T07:33:35Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.