JudgePanel: A Compact Judge with Panel Deliberation via Adaptive Multi-Reward Reinforcement Learning

• URL: http://arxiv.org/abs/2608.29168v1
• Date: Sat, 29 Aug 2026 09:33:49 GMT
• Title: JudgePanel: A Compact Judge with Panel Deliberation via Adaptive Multi-Reward Reinforcement Learning
• Abstract summary: Single-model judges are limited by their inherent model biases.<n>Multi-agent evaluation protocols that mitigate this through diverse deliberation are prohibitively expensive at inference time.<n>We propose textbfmodelname, which equips a compact underlineJudge model with multi-agent underlinePanel deliberation capability.
• Score: 5.81106044690391
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model biases, while multi-agent evaluation protocols that mitigate this through diverse deliberation are prohibitively expensive at inference time. To this end, we propose \textbf{\modelname}, which equips a compact \underline{Judge} model with multi-agent \underline{Panel} deliberation capability. Specifically, we first train on panel deliberation traces from an ensemble of strong evaluators, capturing structured patterns of discussion, disagreement, and resolution. To further improve judgment quality beyond SFT, we introduce \textit{AdaReward}, an adaptive multi-reward RL algorithm that dynamically rebalances reward component weights as different objectives saturate at different rates during RL training. For practical deployment, we further design a lightweight domain specialization module for rapid adaptation to new evaluation domains with few hundred labeled samples. As a result, (i) \textit{Novel}: the first framework to equip a single compact judge with multi-agent panel deliberation capability at single-model inference cost; (ii) \textit{Effective \& Reliable}: JudgePanel with a 14B backbone outperforms judge-specialized models up to 70B across four evaluation benchmarks, demonstrates strong position consistency, and rapidly specializes to new domains with few hundred samples.

Related papers

• Sharding Prevents LLM Oversight Failures and Adversarial Exploitation [45.380649793616705]
  Sharding partitions requirements into smaller groups, assigns each group to a separate call, and aggregates the verdicts.<n>We find that a sharded weaker judge can outperform a more capable holistic judge and match that judge even when the latter receives the panel's full budget.
  arXiv  Detail & Related papers  (2026-08-05T18:21:21Z)
• Selective Ensemble Based on Preference-Directed Multi-Objective Bandits [90.75513823660775]
  We formalize a sequential decision problem under partially specified linear preferences.<n>We then propose a preference-directed upper confidence bound (PrefUCB) algorithm.<n> Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate our method.
  arXiv  Detail & Related papers  (2026-06-20T07:52:46Z)
• TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models [76.92890872338235]
  Large-scale pre-trained Vision-Language models (VLMs) exhibit strong zero-shot generalization, yet are highly vulnerable to imperceptible adversarial perturbations.<n>To enhance robustness without requiring downstream task-specific retraining, we propose TAME, a novel test-time defense.
  arXiv  Detail & Related papers  (2026-05-17T18:07:08Z)
• You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass [40.11359880802771]
  We present a discriminative multimodal reward model that scores all candidate responses in a single forward pass.<n>The multi-response design also yields up to $Ntimes$ wall-clock speedup and FLOPs reduction over conventional single-response scoring.<n>Our model outperforms existing larger generative and discriminative reward models.
  arXiv  Detail & Related papers  (2026-04-13T04:02:03Z)
• CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling [61.75914342638658]
  CDRRM (Contrast-Driven Reward Model) is a framework for high-quality rubric generation and guided preference judgment.<n>This work offers a scalable, interpretable, and data-efficient path for reward modeling.
  arXiv  Detail & Related papers  (2026-03-09T07:15:23Z)
• Judge Model for Large-scale Multimodality Benchmarks [11.960445424565895]
  We propose a dedicated multimodal Judge Model to provide reliable, explainable evaluation across a diverse suite of tasks.<n>Our framework aggregates multimodal judgments, analyzes the quality and reasoning consistency of model outputs, and generates diagnostic feedback.<n>Results show strong alignment between the Judge Model and human scores, demonstrating its potential as a scalable, interpretable evaluation pipeline.
  arXiv  Detail & Related papers  (2026-01-03T07:17:17Z)
• Approximating Human Preferences Using a Multi-Judge Learned System [35.18016233072556]
  We propose a framework for modeling diverse, persona-based preferences by learning to aggregate outputs from multiple rubric-conditioned judges.<n>Our contributions include a persona-based method for synthesizing preference labels at scale and two distinct implementations of our aggregator.
  arXiv  Detail & Related papers  (2025-10-29T18:32:53Z)
• Think Twice: Branch-and-Rethink Reasoning Reward Model [32.70732791642558]
  We introduce branch-and-rethink (BR-RM), a two-turn RM that transfers the think-twice principle to reward modeling.<n>We train with GRPO-style reinforcement learning over structured two-turn traces using a simple binary outcome reward with strict format checks.<n>By converting all-at-oncescoringintofocused, second-lookreasoning, BR-RMreducesjudgmentdiffusionandimproves sensitivity to subtle yet consequential errors.
  arXiv  Detail & Related papers  (2025-10-27T17:58:07Z)
• Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles [20.7718577645105]
  Multimodal large language models (MLLMs) are increasingly used to evaluate text-to-image (TTI) generation systems.<n>These "judge" models often suffer from biases, overconfidence, and inconsistent performance across diverse image domains.<n>We propose a new multimodal-aware method called Multimodal Mixture-of-Bayesian Prompt Ensembles (MMB)
  arXiv  Detail & Related papers  (2025-09-10T17:06:47Z)
• UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality [52.49062565901046]
  Reinforcement Learning from Human Feedback (RLHF) has become a cornerstone for aligning large language models with human values.<n>Existing approaches struggle to capture the multi-dimensional, distributional nuances of human preferences.<n>We introduce Utility-Conditioned Multi-Objective Alignment (UC-MOA), a novel framework that overcomes these limitations.
  arXiv  Detail & Related papers  (2025-03-10T09:52:42Z)
• Aligning Large Language Models by On-Policy Self-Judgment [49.31895979525054]
  Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model (RM) for on-policy learning. We present a novel alignment framework, SELF-JUDGE, that does on-policy learning and is parameter efficient. We show that the rejecting sampling by itself can improve performance further without an additional evaluator.
  arXiv  Detail & Related papers  (2024-02-17T11:25:26Z)
• Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning [58.2091760793799]
  We propose a novel contrastive prototype learning with augmented embeddings (CPLAE) model. With a class prototype as an anchor, CPL aims to pull the query samples of the same class closer and those of different classes further away. Extensive experiments on several benchmarks demonstrate that our proposed CPLAE achieves new state-of-the-art.
  arXiv  Detail & Related papers  (2021-01-23T13:22:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.