An Empirical Investigation of Practical LLM-as-a-Judge Improvement Techniques on RewardBench 2

• URL: http://arxiv.org/abs/2604.13717v1
• Date: Wed, 15 Apr 2026 10:52:33 GMT
• Title: An Empirical Investigation of Practical LLM-as-a-Judge Improvement Techniques on RewardBench 2
• Abstract summary: We present techniques that improve GPT-5.4 judge accuracy on RewardBench 2 without any finetuning.<n>Two techniques account for nearly all available gains: task-specific criteria injection and ensemble scoring.<n>Three further techniques did not reliably improve on criteria + ensembling at comparable cost.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-as-a-judge, using a language model to score or rank candidate responses, is widely used as a scalable alternative to human evaluation in RLHF pipelines, benchmarking, and application layer evaluations (evals). However, judgment reliability depends heavily on prompting and aggregation strategy. We present an empirical investigation of practical, drop-in techniques that improve GPT-5.4 judge accuracy on RewardBench 2 without any finetuning. Two techniques account for nearly all available gains: task-specific criteria injection (+3.0pp at negligible cost) and ensemble scoring (+9.8pp at 5x cost). Combined, they reach 83.6% accuracy, +11.9pp over the 71.7% baseline. Our investigation also covers three further techniques (calibration context, adaptive model escalation, and soft blending) which did not reliably improve on criteria + ensembling at comparable cost. Cheaper model tiers benefit disproportionately from ensembling: GPT-5.4 mini with k=8 achieves 79.2% at 1.2x baseline cost, and GPT-5.4 nano with k=8 reaches 71.4% at 0.4x baseline cost, making high-accuracy LLM judges accessible at low cost.

Related papers

• Small Language Models as Judges for Rubric-Based Reinforcement Learning [57.707881387886516]
  We study whether smaller language models can serve as efficient and reliable rubric-based judges.<n>We compare three ways of extracting criterion-level judgments from small models: Generative verdicts, Yes/No Logprob margins, and Probe judges.<n>Across both datasets, the Qwen3-1.7B Probe judge achieves the strongest criterion-level agreement among these methods.
  arXiv  Detail & Related papers  (2026-08-30T20:00:17Z)
• AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling [3.2665457005470504]
  LLM judges are widely used to evaluate agentic tool-calling systems, yet their reliability on structured, dependency-driven systems remains largely unexamined.<n>We present AgentJudgeBench, the first benchmark to systematically study LLM-as-a-judge reliability for agentic tool-calling over workflow DAGs.
  arXiv  Detail & Related papers  (2026-08-27T05:20:22Z)
• InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy [0.9924185669370708]
  InvestPhilBench is a dynamic benchmark spanning eight cognitive tiers.<n>The v0.6 release comprises 118 primary-source-verified investment principle cards.<n>For reproducible scoring at scale we introduce the Benchmark Automated Scoring Pipeline.
  arXiv  Detail & Related papers  (2026-06-24T15:53:20Z)
• Step-wise Rubric Rewards for LLM Reasoning [72.17879367869503]
  Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models.<n>We find 18.2% of steps in correct responses are wrong yet positively rewarded.<n>Across six mathematical reasoning benchmarks, SRaR improves average accuracy over RaR by 3.57 points.
  arXiv  Detail & Related papers  (2026-05-17T07:08:14Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration [0.0]
  Training tool-calling agents with reinforcement learning on multi-turn tasks remains challenging.<n>We present the first application of MT-GRPO and GTPO for training a tool-calling agent on realistic customer service tasks.
  arXiv  Detail & Related papers  (2026-04-03T08:36:03Z)
• REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge [83.2858110368572]
  We propose textbfREAL (underlineREgression-underlineAware Reinforcement underlineLThought), a principled RL framework designed to optimize regression rewards.<n>We show that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods.
  arXiv  Detail & Related papers  (2026-03-17T21:19:08Z)
• How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs [49.61011897610774]
  How2Everything is a framework to evaluate and improve goal-conditioned procedure generation.<n>Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics.<n>How2Score is an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal.
  arXiv  Detail & Related papers  (2026-02-09T15:47:14Z)
• EvasionBench: Detecting Evasive Answers in Financial Q&A via Multi-Model Consensus and LLM-as-Judge [8.50639201265868]
  We introduce EvasionBench, comprising 30,000 training samples and 1,000 human-annotated test samples.<n>We mine boundary cases where two strong annotators conflict, using a judge to resolve labels.<n>Our trained model Eva-4B (4B parameters) achieves 81.3 percent accuracy, outperforming its base by 25 percentage points.
  arXiv  Detail & Related papers  (2026-01-14T04:26:43Z)
• LLM-as-a-Judge for Scalable Test Coverage Evaluation: Accuracy, Operational Reliability, and Cost [1.7133809948345597]
  We present a production-ready framework for evaluating acceptance tests with structured evaluations.<n>We provide the first comprehensive analysis spanning accuracy, operational reliability, and cost.<n>We release the dataset, framework, and code to support and deployment.
  arXiv  Detail & Related papers  (2025-12-01T03:19:33Z)
• Humains-Junior: A 3.8B Language Model Achieving GPT-4o-Level Factual Accuracy by Directed Exoskeleton Reasoning [0.0]
  Humans-Junior is a 3.8B model that matches GPT-4o on the FACTS Grounding public subset within a $pm 5$ pp equivalence margin.<n>Our approach combines minimal directed "Exoskeleton Reasoning" scaffolds with behavioral fine-tuning that teaches protocol compliance.
  arXiv  Detail & Related papers  (2025-10-29T20:12:36Z)
• Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance [0.0]
  Reward-model training is the cost bottleneck in modern Reinforcement Learning Human Feedback (RLHF) pipelines.<n>In the proposed method, a frozen, instruction-tuned 7B LLM is augmented with only a one line and a rank-16 LoRA adapter.<n>The plug-and-play judge 96.2% achieves accuracy on RewardBench, outperforming specialized reward networks ranging from 27B to 70B parameters.
  arXiv  Detail & Related papers  (2025-06-06T05:18:54Z)
• How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments [83.78240828340681]
  GAMA($gamma$)-Bench is a new framework for evaluating Large Language Models' Gaming Ability in Multi-Agent environments.<n>$gamma$-Bench includes eight classical game theory scenarios and a dynamic scoring scheme specially designed to assess LLMs' performance.<n>Our results indicate GPT-3.5 demonstrates strong robustness but limited generalizability, which can be enhanced using methods like Chain-of-Thought.
  arXiv  Detail & Related papers  (2024-03-18T14:04:47Z)
• How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts [54.07541591018305]
  We present MAD-Bench, a benchmark that contains 1000 test samples divided into 5 categories, such as non-existent objects, count of objects, and spatial relationship. We provide a comprehensive analysis of popular MLLMs, ranging from GPT-4v, Reka, Gemini-Pro, to open-sourced models, such as LLaVA-NeXT and MiniCPM-Llama3. While GPT-4o achieves 82.82% accuracy on MAD-Bench, the accuracy of any other model in our experiments ranges from 9% to 50%.
  arXiv  Detail & Related papers  (2024-02-20T18:31:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.