RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges

• URL: http://arxiv.org/abs/2608.24231v1
• Date: Tue, 25 Aug 2026 08:35:04 GMT
• Title: RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges
• Abstract summary: RecurSE pairs a trainable judge with a synchronized policy-copy checker that audits the judge's reasoning against meta-rubrics to supply a scalar process reward.<n>We show that RecurSE achieves consistent generalization gains across held-out medical, pairwise, summarization, and professional benchmarks.
• Score: 48.831446371402414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-as-judge is essential for evaluating open-ended text and steering post-training, yet improving the judge itself typically relies on expensive annotations, reward models, or distillation from stronger teachers. In this work, we eliminate external gold supervision from the RL training reward: the model's own evaluative capability generates learning signals for its optimization -- a closed-loop setting of bounded recursive self-improvement (RSI) termed Recursive Self-Evaluation (RecurSE). We study two central questions: when can self-improvement occur, and when must it stop? First, RecurSE pairs a trainable judge evaluating candidate responses under per-rule rubrics (Pass 1) with a synchronized policy-copy checker that audits the judge's reasoning against meta-rubrics to supply a scalar process reward (Pass 2). To enable learning, interface decoupling structurally isolates the checker's scalar score from the judge's verdict tokens, eliminating a degenerative token-copying shortcut that inflates self-assigned rewards. Second, because unanchored recursive learning is inherently bounded, Pairwise Advantage Validity (PAV) serves as an unbiased validation monitor that jointly tracks judge accuracy and checker fidelity to reliably identify the optimal early-stopping window. Across Qwen3.5-9B, Gemma-4-E4B-it, and Qwen3.6-27B, RecurSE achieves consistent generalization gains across held-out medical, pairwise, summarization, and professional benchmarks. Ablations demonstrate that synchronized judge-checker co-evolution outperforms frozen checkers, external meta-judges, self-consistency, and scaled teacher distillation. Furthermore, preference pairs curated by our judge effectively enhance downstream policy alignment. Bounded RSI for LLM-as-judge is thus viable when self-produced reward validity is explicitly decoupled and monitored.

Related papers

• SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute [62.3458279176813]
  Self-Verifying Refinement is an oracle-free multi-turn reinforcement learning framework.<n>It learns to use self-verification as a compute-control policy.<n>It achieves a macro-average accuracy of 0.563 with only 2.99 inference turns on average.
  arXiv  Detail & Related papers  (2026-07-30T16:20:58Z)
• LOTAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning [12.013691798765594]
  LOTAPO is a self-generated process-supervision method based on backward leave-one-turn attribution.<n>For each search turn, LOTAPO replaces the turn and its retrieval observation with a fixed [DELETE] placeholder.<n>This Answer-Likelihood Gain estimates the turn's contribution while preserving all downstream interactions.
  arXiv  Detail & Related papers  (2026-07-15T06:55:28Z)
• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation [60.55792673956761]
  We propose textbfRubric-Conditioned Self-Distillation, a framework that incorporates rubrics as structured, fine-grained feedback for on-policy self-distillation.<n>We evaluate on a diverse suite of science reasoning benchmarks and results show that rubric-conditioned self-distillation effectively converts rubric-level criteria into token-level guidance.
  arXiv  Detail & Related papers  (2026-06-17T17:54:04Z)
• Is Fairness Truly Fair? Towards Reliable Lipschitz Fairness in Multi-Task Learning via Fixed-\texorpdfstring{$δ$}{delta} Alignment [0.32792728203318305]
  Lipschitz-style individual fairness formalizes the idea that semantically similar examples should receive similar predictions.<n>This paper identifies threshold confounding: when the auditing tolerance is derived from each model's own representation distances, different algorithms are compared under different semantic thresholds.<n>We propose textbfReLiF, a reliability-aware framework that separates evaluation-time fixed-$$ auditing from training-time controlled regularization.
  arXiv  Detail & Related papers  (2026-06-09T09:36:54Z)
• AI Alignment via Incentives and Correction [14.986111703734222]
  We study AI alignment through the lens of law-and-economics models of deterrence and enforcement.<n>A solver may benefit from producing a persuasive but incorrect answer, hiding uncertainty, or exploiting spurious shortcuts.<n>We formalize this interaction in a two-agent model in which a principal chooses rewards over joint correction outcomes.
  arXiv  Detail & Related papers  (2026-05-02T23:28:02Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.