No Single Best Model for Diversity: Learning a Router for Sample Diversity

• URL: http://arxiv.org/abs/2604.02319v1
• Date: Thu, 02 Apr 2026 17:58:37 GMT
• Title: No Single Best Model for Diversity: Learning a Router for Sample Diversity
• Abstract summary: In this paper, we study methods to elicit a comprehensive set of valid responses.<n>We introduce textbfdiversity coverage, a metric that measures the total quality scores assigned to each textbfunique answer.<n>Per each prompt, there exists a model that outperforms all other models significantly at generating a diverse answer set.
• Score: 69.53166985556759
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When posed with prompts that permit a large number of valid answers, comprehensively generating them is the first step towards satisfying a wide range of users. In this paper, we study methods to elicit a comprehensive set of valid responses. To evaluate this, we introduce \textbf{diversity coverage}, a metric that measures the total quality scores assigned to each \textbf{unique} answer in the predicted answer set relative to the best possible answer set with the same number of answers. Using this metric, we evaluate 18 LLMs, finding no single model dominates at generating diverse responses to a wide range of open-ended prompts. Yet, per each prompt, there exists a model that outperforms all other models significantly at generating a diverse answer set. Motivated by this finding, we introduce a router that predicts the best model for each query. On NB-Wildchat, our trained router outperforms the single best model baseline (26.3% vs $23.8%). We further show generalization to an out-of-domain dataset (NB-Curated) as well as different answer-generation prompting strategies. Our work lays foundation for studying generating comprehensive answers when we have access to a suite of models.

Related papers

• RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences [3.498051445282382]
  This work focuses on how to evaluate the generalizability of reward models.<n>We introduce RMGAP, a benchmark comprising 1,097 instances across Chat, Writing, Reasoning, and Safety domains.<n>Even the best RM achieves only 49.27% Best-of-N accuracy, highlighting considerable room for improvement in reward model generalization.
  arXiv  Detail & Related papers  (2026-05-03T11:45:08Z)
• Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models [78.68818219506313]
  This paper describes a multi-answer reinforcement learning approach for training LMs to perform distributional reasoning over multiple answers during inference.<n>Across question-answering, medical diagnostic, and coding benchmarks, we observe improved diversity, coverage, and set-level calibration scores compared to single answer trained baselines.
  arXiv  Detail & Related papers  (2026-03-25T22:20:25Z)
• The Majority is not always right: RL training for solution aggregation [53.1050856072799]
  We train an aggregator model to review, reconcile, and synthesize a final, correct answer.<n>A key ingredient is careful balancing of easy and hard training examples.<n>We find our method, AggLM, outperforms both strong rule-based and reward-model baselines.
  arXiv  Detail & Related papers  (2025-09-08T16:39:38Z)
• SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions [17.12876663526222]
  Large language models (LLMs) are increasingly evaluated on single-answer multiple-choice tasks.<n>Many real-world problems require identifying all correct answers from a set of options.<n>We introduce SATA-BENCH, the first dedicated benchmark for evaluating LLMs on Select All That Apply questions.
  arXiv  Detail & Related papers  (2025-05-31T17:14:21Z)
• Model-Based Simulation for Optimising Smart Reply [3.615981646205045]
  Smart Reply (SR) systems present a user with a set of replies, of which one can be selected in place of having to type out a response. Previous work has focused largely on post-hoc diversification, rather than explicitly learning to predict sets of responses. We present a novel method SimSR, that employs model-based simulation to discover high-value response sets.
  arXiv  Detail & Related papers  (2023-05-26T12:04:33Z)
• Getting MoRE out of Mixture of Language Model Reasoning Experts [71.61176122960464]
  We propose a Mixture-of-Reasoning-Experts (MoRE) framework that ensembles diverse specialized language models. We specialize the backbone language model with prompts optimized for different reasoning categories, including factual, multihop, mathematical, and commonsense reasoning. Our human study confirms that presenting expert predictions and the answer selection process helps annotators more accurately calibrate when to trust the system's output.
  arXiv  Detail & Related papers  (2023-05-24T02:00:51Z)
• SRQA: Synthetic Reader for Factoid Question Answering [21.28441702154528]
  We introduce a new model called SRQA, which means Synthetic Reader for Factoid Question Answering. This model enhances the question answering system in the multi-document scenario from three aspects. We perform SRQA on the WebQA dataset, and experiments show that our model outperforms the state-of-the-art models.
  arXiv  Detail & Related papers  (2020-09-02T13:16:24Z)
• Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets [37.02260241742407]
  We study the test sets of three popular open-domain benchmark datasets. We find that 60-70% of test-time answers are also present somewhere in the training sets. We also find that 30% of test-set questions have a near-duplicate paraphrase in their corresponding training sets.
  arXiv  Detail & Related papers  (2020-08-06T13:17:43Z)
• ManyModalQA: Modality Disambiguation and QA over Diverse Inputs [73.93607719921945]
  We present a new multimodal question answering challenge, ManyModalQA, in which an agent must answer a question by considering three distinct modalities. We collect our data by scraping Wikipedia and then utilize crowdsourcing to collect question-answer pairs.
  arXiv  Detail & Related papers  (2020-01-22T14:39:28Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.