Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs

• URL: http://arxiv.org/abs/2510.17472v2
• Date: Thu, 23 Oct 2025 14:03:17 GMT
• Title: Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs
• Abstract summary: We present a unified framework for certifiable inference in large language models.<n>We show that majority voting provides a statistical certificate of self-consistency.<n>We further prove that label-free post-training methods such as TTRL implicitly sharpen the answer distribution.
• Score: 4.829906774017035
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent advances such as self-consistency and test-time reinforcement learning (TTRL) improve the reliability of large language models (LLMs) without additional supervision, yet their underlying mechanisms and statistical guarantees remain poorly understood. We present a unified framework for certifiable inference in LLMs, showing that majority voting provides a statistical certificate of self-consistency: under mild assumptions, the aggregated answer coincides with the mode of the model's terminal distribution with high probability. We derive finite-sample and anytime-valid concentration bounds that quantify this confidence, and introduce the Martingale Majority Certificate (MMC), a sequential stopping rule that adaptively determines when sufficient samples have been drawn. We further prove that label-free post-training methods such as TTRL implicitly sharpen the answer distribution by exponentially tilting it toward its mode, thereby reducing the number of samples required for certification. Building on this insight, we propose new post-training objectives that explicitly optimise this trade-off between sharpness and bias. Together, these results explain and connect two central test-time scaling strategies, self-consistency and TTRL, within a single statistical framework for label-free, certifiable reliability in reasoning LLMs.

Related papers

• IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals [2.623527845999313]
  We argue that reliable factuality control can be achieved through introspective signals derived from the model itself.<n>IntroConformal is a training-free Conformal Risk Control framework that provides finite-sample, distribution-free factuality guarantees.
  arXiv  Detail & Related papers  (2026-09-01T15:09:56Z)
• The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection [4.921591758479804]
  Statistical tools for detecting training-data membership exist, but have been validated almost exclusively in controlled academic regimes.<n>We identify two under-studied failure modes: distribution shift and scale constraints.<n>Across 335 evaluations, only 199 yield correct outcomes.
  arXiv  Detail & Related papers  (2026-06-02T08:21:22Z)
• A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering [19.55210880950831]
  Expected measures whether a model's predicted confidence aligns with its empirical accuracy, and is central to the reliable deployment of large language models (LLMs)<n>We introduce Sem-ECE, a calibration evaluation framework for open-ended question answering (QA)<n>We study two estimators within framework: Sem$$-ECE, the same-sample self-consistency score, and Sem$$-ECE, a held-out variant that separates answer selection from confidence evaluation.
  arXiv  Detail & Related papers  (2026-05-08T19:53:49Z)
• CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency [10.34950275095264]
  We study anytime-valid certification of a prespecified target answer as unique mode of the model's response distribution.<n>We propose the Certification by Intersection-union Testing with E-processes (CITE) algorithm, which provably controls false certification at any prescribed level.<n>We also prove an category-set-size-free stopping-time rate, establish matching minimax lower bounds up to constants in the main regime, and extend the construction to confidence-weighted voting.
  arXiv  Detail & Related papers  (2026-05-07T08:41:51Z)
• Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity [70.60312607268871]
  FedQual is a quality-aware Fed-LDL framework with two mechanisms.<n>We construct four new Fed-LDL benchmarks with controlled annotation quality disparity.
  arXiv  Detail & Related papers  (2026-05-06T12:20:20Z)
• Process Supervision of Confidence Margin for Calibrated LLM Reasoning [52.373121066425455]
  Scaling test-time computation with reinforcement learning (RL) has emerged as a reliable path to improve large language models (LLM) reasoning ability.<n>Yet, outcome-based reward often incentivizes models to be overconfident, leading to hallucinations, unreliable confidence-based control, and unnecessary compute allocation.<n>We introduce Reinforcement Learning with Confidence Margin (textbfRLCM), a calibration-aware RL framework that jointly optimize correctness and confidence reliability.
  arXiv  Detail & Related papers  (2026-04-25T14:40:13Z)
• FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning [49.65751420291115]
  Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps.<n>We propose FACT-E, a causality-inspired framework for evaluating CoT quality.<n>We show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learnings.
  arXiv  Detail & Related papers  (2026-04-12T15:35:08Z)
• What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time [57.533031432715084]
  Test-Time Reinforcement Learning (TTRL) enables Large Language Models (LLMs) to enhance reasoning capabilities on unlabeled test streams.<n>Existing TTRL methods rely exclusively on positive pseudo-labeling strategies.<n>We propose SCRL, a robust test-time reinforcement learning framework that effectively mitigates label noise amplification.
  arXiv  Detail & Related papers  (2026-03-20T11:47:12Z)
• BD-Merging: Bias-Aware Dynamic Model Merging with Evidence-Guided Contrastive Learning [2.8115115690134744]
  Model Merging (MM) has emerged as a scalable paradigm for multi-task learning (MTL)<n>Despite recent progress, the reliability of MM under test-time distribution shift remains insufficiently understood.<n>We present BD-Merging, a bias-aware unsupervised model merging framework that explicitly models uncertainty to achieve adaptive reliability under distribution shift.
  arXiv  Detail & Related papers  (2026-03-04T10:27:56Z)
• STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction [78.0692157478247]
  We propose STAR, a framework that bridges data-driven STatistical expectations with knowledge-driven Agentic Reasoning.<n>We show that STAR consistently outperforms all baselines on both score-based and rank-based metrics.
  arXiv  Detail & Related papers  (2026-02-12T16:30:07Z)
• Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance [51.532841645285835]
  We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution.<n>We show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution.<n>We introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure.
  arXiv  Detail & Related papers  (2026-02-10T19:08:40Z)
• Test-Time Adaptation for Tactile-Vision-Language Models [6.918033886034668]
  Tactile-vision-language (TVL) models are increasingly deployed in real-world robotic and multimodal perception tasks.<n>Existing test-time adaptation methods provide filtering in unimodal settings but lack explicit treatment of modality-wise reliability under cross-modal shifts.<n>We propose a reliability-aware framework that estimates per-modality reliability from prediction uncertainty and perturbation-based responses.
  arXiv  Detail & Related papers  (2026-01-31T02:26:01Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classification [29.336193842747832]
  We introduce Self-Filtered Distillation, a framework specifically tailored for patent classification.<n>It treats LLM-generated rationales as trust signals rather than ground-truth supervision.<n>Our method outperforms label-based learning and conventional distillation in accuracy, stability, and interpretability.
  arXiv  Detail & Related papers  (2025-10-06T22:50:01Z)
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning [62.452350134196934]
  FaithCoT-Bench is a unified benchmark for instance-level CoT unfaithfulness detection.<n>Our framework formulates unfaithfulness detection as a discriminative decision problem.<n>FaithCoT-Bench sets a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.
  arXiv  Detail & Related papers  (2025-10-05T05:16:54Z)
• Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation [63.49409574310576]
  Large language models (LLMs) exhibit overconfidence, assigning high confidence scores to incorrect predictions.<n>We introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation.<n>Our code and all baselines used in the paper are available on GitHub.
  arXiv  Detail & Related papers  (2025-08-16T13:29:35Z)
• Conformal P-Value in Multiple-Choice Question Answering Tasks with Provable Risk Control [0.0]
  This study introduces a significance testing-enhanced conformal prediction (CP) framework to improve trustworthiness of large language models (LLMs) in multiple-choice question answering (MCQA)<n>CP provides statistically rigorous marginal coverage guarantees for prediction sets, and significance testing offers established statistical rigor.<n>This work establishes a statistical framework for trustworthy LLM deployment in high-stakes QA applications.
  arXiv  Detail & Related papers  (2025-08-07T16:46:47Z)
• Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guarantees [5.09580026885155]
  We propose a frequency-based uncertainty quantification method under black-box settings.<n>Our approach involves multiple independent samplings of the model's output distribution for each input.<n>We show that frequency-based PE outperforms logit-based PE in distinguishing between correct and incorrect predictions.
  arXiv  Detail & Related papers  (2025-08-07T16:22:49Z)
• Conformal Calibration of Statistical Confidence Sets [1.8840155706520811]
  We introduce two novel methods, TRUST and TRUST++, for calibrating confidence sets to achieve distribution-free conditional coverage.<n>We demonstrate that our methods outperform existing approaches, particularly in small-sample regimes.
  arXiv  Detail & Related papers  (2024-11-28T20:45:59Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Conformal Prediction for Federated Uncertainty Quantification Under Label Shift [57.54977668978613]
  Federated Learning (FL) is a machine learning framework where many clients collaboratively train models. We develop a new conformal prediction method based on quantile regression and take into account privacy constraints.
  arXiv  Detail & Related papers  (2023-06-08T11:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.