Improving LLMs via Validator-to-Generator Alignment

• URL: http://arxiv.org/abs/2607.02668v1
• Date: Thu, 02 Jul 2026 18:00:39 GMT
• Title: Improving LLMs via Validator-to-Generator Alignment
• Abstract summary: We introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency.<n>We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally.
• Score: 42.390756533626686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are inconsistent: varying prompts or including unrelated information can lead to unexpected changes in model outputs. The generator-validator (G-V) gap is one manifestation of this phenomenon, where LLMs generate responses that they then deem as invalid if re-queried to validate them. In this work, we introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency. Specifically, generators often assign low likelihood to valid strings simply because those strings are a priori unlikely, which makes naive notions of G-V consistency unworkable. We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally. Our method, \emph{\FCPAname} (\FCPA), is a training objective implementing frequency-corrected G-V consistency for real-world LLMs. Our experimental results show that training with \FCPA{} substantially improves both G-V consistency and generator performance over prior methods, with gains of up to $+27$pp in Pearson correlation on IFEval and HumanEval, while preserving validator quality across all evaluated tasks.

Related papers

• FUSE: Ensembling Verifiers with Zero Labeled Data [11.563701622899295]
  Fully Unsupervised Score Ensembling (FUSE) is a method for improving verification quality by ensembling verifiers without access to ground truth correctness labels.<n>Despite requiring zero ground truth labels, FUSE typically matches or improves upon semi-supervised alternatives in test-time scaling experiments.
  arXiv  Detail & Related papers  (2026-04-20T17:40:33Z)
• VIGOR+: Iterative Confounder Generation and Validation via LLM-CEVAE Feedback Loop [14.309475903975441]
  Recent advances leverage Large Language Models to generate plausible hidden confounders based on domain knowledge.<n>We propose VIGOR+, a novel framework that closes the loop between LLM-based confounder generation and CEVAE-based statistical validation.<n>We formalize the feedback mechanism, prove convergence properties under mild assumptions, and provide a complete algorithmic framework.
  arXiv  Detail & Related papers  (2025-12-22T12:48:29Z)
• EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI [36.91800117379075]
  EVADE is a framework for generating and validating explanations to detect errors using large language models.<n>HLV arises when multiple labels are valid for the same instance, making it difficult to separate annotation errors from plausible variation.
  arXiv  Detail & Related papers  (2025-11-12T03:49:05Z)
• Efficient Real-time Refinement of Language Model Text Generation [65.1937138219008]
  Large language models (LLMs) have shown remarkable performance across a wide range of natural language tasks.<n>A critical challenge remains in that they sometimes generate factually incorrect answers.<n>We propose Streaming-VR, a novel approach designed to enhance the efficiency of verification and refinement of LLM outputs.
  arXiv  Detail & Related papers  (2025-01-14T03:59:48Z)
• Generative Verifiers: Reward Modeling as Next-Token Prediction [29.543787728397643]
  We propose training verifiers using the ubiquitous next-token prediction objective, jointly on verification and solution generation.<n>Compared to standard verifiers, such generative verifiers (GenRM) can benefit from several advantages of LLMs.<n>We observe improvements of 28% $rightarrow$ 44.6% on MATH, and 37.9% $rightarrow$ 53.5% on MMLU abstract algebra.
  arXiv  Detail & Related papers  (2024-08-27T17:57:45Z)
• Validating LLM-Generated Programs with Metamorphic Prompt Testing [8.785973653167112]
  Large Language Models (LLMs) are increasingly integrated into the software development lifecycle. This paper proposes a novel solution called metamorphic prompt testing to address these challenges. Our evaluation on HumanEval shows that metamorphic prompt testing is able to detect 75 percent of the erroneous programs generated by GPT-4, with a false positive rate of 8.6 percent.
  arXiv  Detail & Related papers  (2024-06-11T00:40:17Z)
• Self-Evaluation Improves Selective Generation in Large Language Models [54.003992911447696]
  We reformulate open-ended generation tasks into token-level prediction tasks. We instruct an LLM to self-evaluate its answers. We benchmark a range of scoring methods based on self-evaluation.
  arXiv  Detail & Related papers  (2023-12-14T19:09:22Z)
• Benchmarking and Improving Generator-Validator Consistency of Language Models [82.73914625520686]
  inconsistency between generating and validating an answer is prevalent in language models (LMs) Even GPT-4, a state-of-the-art LM, is GV-consistent only 76% of the time. We find that this approach improves GV-consistency of Alpaca-30B from 60% to 93%.
  arXiv  Detail & Related papers  (2023-10-03T07:23:22Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Zero-Shot Logit Adjustment [89.68803484284408]
  Generalized Zero-Shot Learning (GZSL) is a semantic-descriptor-based learning technique. In this paper, we propose a new generation-based technique to enhance the generator's effect while neglecting the improvement of the classifier. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative zero-shot learning frameworks.
  arXiv  Detail & Related papers  (2022-04-25T17:54:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.