GenSelect: A Generative Approach to Best-of-N

• URL: http://arxiv.org/abs/2507.17797v1
• Date: Wed, 23 Jul 2025 15:22:51 GMT
• Title: GenSelect: A Generative Approach to Best-of-N
• Abstract summary: We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates.<n>For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect.
• Score: 46.74684914760564
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or pairwise comparisons. However, pointwise methods underutilize LLMs' comparative abilities, while pairwise methods scale inefficiently with larger sampling budgets. We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates. This leverages LLMs' comparative strengths while scaling efficiently across parallel sampling budgets. For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect, outperforming existing scoring approaches with simple prompting.

Related papers

• Data Efficient Sample Selection for In-Context Learning [7.910984819642885]
  In-context learning (ICL) aids large language models (LLMs) to adapt to new tasks without need for fine-tuning.<n>Existing approaches for selection do not model the complex relationship between ICL samples and downstream LLM performance.<n>We introduce DearICL, a new framework that models demonstration example selection as a subset ranking problem.
  arXiv  Detail & Related papers  (2026-09-06T15:23:37Z)
• Mixture of Complementary Agents for Robust LLM Ensemble [28.210955621253863]
  Multi-AI collaboration, such as ensembling or debating large language models (LLMs), is a promising paradigm for aggregating information and boosting performance.<n>Existing approaches primarily focus on accuracy (picking the strongest models) or diversity (ensuring variety)<n>We reframe proposer selection as a selection problem akin to feature selection, where the value of an LLM lies in its complementarity with others.
  arXiv  Detail & Related papers  (2026-05-21T21:29:05Z)
• Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences [7.715856473694668]
  We introduce Duel-Evolve, an evolutionary algorithm that replaces external scalar rewards with pairwise preferences elicited from the same LLM used to generate candidates.<n>Duel-Evolve aggregates noisy candidate comparisons via a Bayesian Bradley-Terry model, yielding uncertainty-aware estimates of candidate quality.<n>We evaluate Duel-Evolve on MathBench, where it achieves 20 percentage points higher accuracy over existing methods and baselines, and on LiveCodeBench, where it improves over comparable iterative methods by over 12 percentage points.
  arXiv  Detail & Related papers  (2026-02-25T05:16:11Z)
• MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning [85.05204262206296]
  Large reasoning models (LRMs) achieve strong performance by producing long chains of thought, but their inference costs are high.<n>Small language models (SLMs) are far more efficient, yet struggle on multi-step reasoning tasks.<n>We propose MentorCollab, an inference-time collaboration method in which an LRM selectively and sparsely guides an SLM, rather than taking over generation.
  arXiv  Detail & Related papers  (2026-02-05T04:58:16Z)
• Learning Generative Selection for Best-of-N [52.88943295436412]
  We show that small reasoning models can acquire strong GenSelect capabilities through targeted reinforcement learning.<n>Our results establish reinforcement learning as a scalable way to unlock strong generative selection in small models.
  arXiv  Detail & Related papers  (2026-02-02T14:21:15Z)
• Access Paths for Efficient Ordering with Large Language Models [7.826046892571884]
  We present the LLM ORDER BY operator as a logical abstraction and study its physical implementations within a unified evaluation framework.<n>We introduce three new designs: an agreement-based batch-size policy, a majority voting mechanism for pairwise sorting, and a two-way external merge sort adapted for LLMs.
  arXiv  Detail & Related papers  (2025-08-30T01:44:36Z)
• Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs [102.48588475875749]
  We introduce Generative Self-Refinement (GSR), a novel parallel test-time scaling framework.<n>GSR generates a set of candidate responses in parallel and then performs self-refinement to synthesize a new superior solution.<n>We show that our method achieves state-of-the-art performance across five mathematical benchmarks.
  arXiv  Detail & Related papers  (2025-08-27T06:51:48Z)
• Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs [45.83245433138508]
  Large language models (LLMs) have rapidly progressed into general-purpose agents capable of solving a broad spectrum of tasks.<n>They apply fixed inference-time compute regardless of task complexity, often overthinking simple problems while underthinking hard ones.<n>This survey presents a comprehensive review of efficient test-time compute strategies, which aim to improve the computational efficiency of LLM reasoning.
  arXiv  Detail & Related papers  (2025-07-02T18:27:42Z)
• Efficient Model Selection for Time Series Forecasting via LLMs [52.31535714387368]
  We propose to leverage Large Language Models (LLMs) as a lightweight alternative for model selection.<n>Our method eliminates the need for explicit performance matrices by utilizing the inherent knowledge and reasoning capabilities of LLMs.
  arXiv  Detail & Related papers  (2025-04-02T20:33:27Z)
• Scalable Best-of-N Selection for Large Language Models via Self-Certainty [65.31658824274894]
  Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models.<n>We propose self-certainty, a novel and efficient metric to estimate response quality without requiring external reward models.<n>Our findings establish self-certainty as a practical and efficient way for improving LLM reasoning capabilities.
  arXiv  Detail & Related papers  (2025-02-25T19:08:07Z)
• Generative Verifiers: Reward Modeling as Next-Token Prediction [29.543787728397643]
  We propose training verifiers using the ubiquitous next-token prediction objective, jointly on verification and solution generation.<n>Compared to standard verifiers, such generative verifiers (GenRM) can benefit from several advantages of LLMs.<n>We observe improvements of 28% $rightarrow$ 44.6% on MATH, and 37.9% $rightarrow$ 53.5% on MMLU abstract algebra.
  arXiv  Detail & Related papers  (2024-08-27T17:57:45Z)
• SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models [8.558834738072363]
  Large language models (LLMs) have been widely adopted due to their remarkable performance across various applications.<n>These individual LLMs show limitations in generalization and performance on complex tasks due to inherent training biases, model size constraints, and the quality or diversity of pre-training datasets.<n>We introduce SelectLLM, which efficiently directs input queries to the most suitable subset of LLMs from a large pool.
  arXiv  Detail & Related papers  (2024-08-16T06:11:21Z)
• Graph-Structured Speculative Decoding [52.94367724136063]
  Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models. We introduce an innovative approach utilizing a directed acyclic graph (DAG) to manage the drafted hypotheses. We observe a remarkable speedup of 1.73$times$ to 1.96$times$, significantly surpassing standard speculative decoding.
  arXiv  Detail & Related papers  (2024-07-23T06:21:24Z)
• BOND: Aligning LLMs with Best-of-N Distillation [63.254031574394965]
  We propose Best-of-N Distillation (BOND), a novel RLHF algorithm that seeks to emulate Best-of-N but without its significant computational overhead at inference time. Specifically, BOND is a distribution matching algorithm that forces the distribution of generations from the policy to get closer to the Best-of-N distribution. We demonstrate the effectiveness of our approach and several design choices through experiments on abstractive summarization and Gemma models.
  arXiv  Detail & Related papers  (2024-07-19T18:38:25Z)
• Leveraging LLMs for Dialogue Quality Measurement [27.046917937460798]
  Large language models (LLMs) show robust zeroshot and few-shot capabilities across NLP tasks. Manipulating factors such as model size, in-context examples, and selection techniques, we examine "chain-of-thought" (CoT) reasoning and label extraction procedures. Our results indicate that LLMs that are suitably fine-tuned and have sufficient reasoning capabilities can be leveraged for automated dialogue evaluation.
  arXiv  Detail & Related papers  (2024-06-25T06:19:47Z)
• Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection [80.63946798650653]
  Decision centers on whether to use a large LLM with better performance or a smaller one with reduced costs. We propose a simpler solution; we use only the uncertainty of the generations of the small LLM as the decision criterion. Our experiments reveal this simple solution optimally balances cost and performance, outperforming existing methods on 25 out of 27 experimental setups.
  arXiv  Detail & Related papers  (2024-05-03T14:38:59Z)
• A distribution-free mixed-integer optimization approach to hierarchical modelling of clustered and longitudinal data [0.0]
  We introduce an innovative algorithm that evaluates cluster effects for new data points, thereby increasing the robustness and precision of this model. The inferential and predictive efficacy of this approach is further illustrated through its application in student scoring and protein expression.
  arXiv  Detail & Related papers  (2023-02-06T23:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.