Boosting Self-Consistency with Ranking

• URL: http://arxiv.org/abs/2606.05054v1
• Date: Wed, 03 Jun 2026 16:12:30 GMT
• Title: Boosting Self-Consistency with Ranking
• Abstract summary: Self-consistency improves large language models by sampling multiple reasoning paths and selecting the most frequent answer.<n>We address this limitation with Ranking-Improved Self-Consistency (RISC), which reformulates answer selection in self-consistency as a ranking problem.
• Score: 56.38798757709555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-consistency improves large language models by sampling multiple reasoning paths and selecting the most frequent answer, but majority voting often fails to recover correct answers that are already present among the samples. We address this limitation with Ranking-Improved Self-Consistency (RISC), which reformulates answer selection in self-consistency as a ranking problem. Instead of relying on a single uncertainty or confidence signal, RISC uses a lightweight LambdaRank model to score candidate answers with five carefully designed features that capture answer frequency, semantic centrality, and reasoning-trace consistency. We evaluate RISC on three datasets under a range of test-time budgets. Across datasets, RISC consistently achieves a better accuracy-efficiency trade-off than standard self-consistency and strong baselines, with particularly large gains on question answering benchmarks. Further analysis shows that the proposed features are individually useful and, more importantly, complementary, highlighting the value of learning to combine multiple informative signals for test-time answer selection.

Related papers

• Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents [0.0]
  We study whether three common families of single-turn UQ methods transfer to an interactive trajectory setting.<n>We find that transfer is often useful but uneven. Token-probability scores are highly sensitive to the choice of aggregator used across turns.<n>Black-box self-consistency is often the strongest UQ family, with trajectory-equivalence and action-set consistency typically ranking highest among its variants.
  arXiv  Detail & Related papers  (2026-08-12T01:39:28Z)
• Beyond Majority Voting: Efficient Best-Of-N with Radial Consensus Score [13.41454380481593]
  Radial Consensus Score (RCS) is a simple, efficient, and training-free method for best-of-N selection.<n>RCS models semantic consensus by computing a weighted Fréchet mean (semantic center) of answer embeddings and ranking candidates by their radial distance to this center.
  arXiv  Detail & Related papers  (2026-04-14T02:02:20Z)
• Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation [47.91529693614168]
  Existing methods are predominantly answer-first, producing confidence only after generating an answer.<n>We study a confidence-first paradigm, where the model outputs its confidence before answering, interpreting this score as the model's probability of answering the question correctly.<n>We propose CoCA, a reinforcement learning framework that jointly optimize confidence calibration and answer accuracy via segmented credit assignment.
  arXiv  Detail & Related papers  (2026-03-06T04:03:13Z)
• Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning [20.371912257758634]
  Self-Consistency improves reasoning reliability through multi-sample aggregation, but incurs substantial inference cost.<n>We propose Reliability-Aware Adaptive Self-Consistency (ReASC), which addresses this limitation by reframing adaptive sampling from response counting to evidence sufficiency.<n>ReASC consistently achieves the best accuracy-cost trade-off compared to existing baselines, yielding improved inference efficiency across model scales from 3B to 27B parameters.
  arXiv  Detail & Related papers  (2026-01-06T12:27:53Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection [71.8243083897721]
  Vision-language models often hallucinate details, generating non-existent objects or inaccurate attributes that compromise output reliability.<n>We present a novel framework that leverages the model's self-consistency between long responses and short answers to generate preference pairs for training.
  arXiv  Detail & Related papers  (2025-09-27T10:37:11Z)
• Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifiers [63.99316853136304]
  Mirror-Critique is a framework that trains a verifier with informative critiques.<n>We deploy a small instruction-tuned model to synthesize high-quality critique data.<n>The resulting Mirror-Verifier is deployed to evaluate candidate solutions by generating multiple critiques per solution.
  arXiv  Detail & Related papers  (2025-09-27T06:50:24Z)
• Mind the Gap: A Closer Look at Tokenization for Multiple-Choice Question Answering with LLMs [16.357595595062946]
  There is no consensus on how to tokenize the space following the colon, often overlooked as a trivial choice.<n>Surprisingly, we are able to recommend one specific strategy -- tokenizing the space together with the answer letter.<n>Our findings underscore the importance of careful evaluation design and highlight the need for standardized, transparent evaluation protocols.
  arXiv  Detail & Related papers  (2025-09-18T14:47:58Z)
• Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering [78.89231943329885]
  Multiple-Choice Question Answering (MCQA) is widely used to evaluate Large Language Models (LLMs)<n>We show that multiple factors can significantly impact the reported performance of LLMs.<n>We analyze whether existing answer extraction methods are aligned with human judgment.
  arXiv  Detail & Related papers  (2025-03-19T08:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.