ComplexConstraints and Beyond: Expert Rubrics for RLVR

• URL: http://arxiv.org/abs/2606.09118v1
• Date: Mon, 08 Jun 2026 07:11:56 GMT
• Title: ComplexConstraints and Beyond: Expert Rubrics for RLVR
• Abstract summary: We present a systematic analysis of expert-curated rubric-based evaluation as an alternative paradigm.<n>We first articulate five design principles for constructing high-quality rubrics, including Maximum Viable Atomicity and intent-aware criterion design.<n>We demonstrate that these expert rubrics are not only better evaluation instruments but also highly effective training signals.
• Score: 1.173565897471208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As LLM capabilities advance rapidly, the evaluation methods used to assess them increasingly lag behind. Traditional benchmarks relied on programmatic verification of narrow, surface-level constraints, but real-world instruction following and agentic tasks demand assessment of nuanced, context-dependent behaviors that resist simple scripted checks. We present a systematic analysis of expert-curated rubric-based evaluation as an alternative paradigm, drawing on empirical evidence from two domains: complex instruction following and enterprise agentic tasks. We first articulate five design principles for constructing high-quality rubrics, including Maximum Viable Atomicity, intent-aware criterion design, and iterative LLM-judge calibration. To validate these principles, we introduce ComplexConstraints, a new expert-curated instruction-following dataset in which each prompt is paired with 10-40 atomic rubric criteria. We demonstrate that these expert rubrics are not only better evaluation instruments but also highly effective training signals: training on approximately 1,000 ComplexConstraints examples yields +15.5% improvement for a 4B-parameter model and +12.2% for a 235B-parameter model on instruction following, while single-epoch RL training on a rubric-graded enterprise environment produces gains that transfer to out-of-distribution benchmarks the model was never trained on (+4.5% BFCL, +7.4% Tau2-Bench, +6.8% Tool-Decathlon). Our findings establish that expert-authored rubrics improve both the measurement and the development of frontier LLM capabilities, serving as effective evaluation and RL training signals.

Related papers

• Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning [1.517713730645682]
  We formalize emphrubric-grounded reinforcement learning (RL): a framework in which the policy is optimized against a structured, multi-criterion reward.<n>We instantiate the framework by deriving rubrics from an Office of Scientific and Technical Information (OSTI)-derived corpus of roughly 100,000 scientific and technical documents.
  arXiv  Detail & Related papers  (2026-05-08T17:48:58Z)
• Creating and Evaluating K-12 GenAI Assessment Graders Through Context Engineering [6.131107680009006]
  The integration of large language models (LLMs) into educational assessment represents a transformative shift in classroom grading practices.<n>This paper examines the theoretical foundations and evaluates an LLM grader that uses commercially available foundation models with context and prompt engineering to score student work against a rubric.
  arXiv  Detail & Related papers  (2026-05-08T16:32:43Z)
• Agentick: A Unified Benchmark for General Sequential Decision-Making Agents [30.028388632526745]
  Agentick is a benchmark for sequential decision-making agents.<n>It provides 37 procedurally generated tasks across six capability categories, four difficulty levels, and five observation modalities.<n>An evaluation spanning 27 configurations and over 90,000 episodes reveals that no single approach dominates.
  arXiv  Detail & Related papers  (2026-05-07T19:12:03Z)
• REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge [83.2858110368572]
  We propose textbfREAL (underlineREgression-underlineAware Reinforcement underlineLThought), a principled RL framework designed to optimize regression rewards.<n>We show that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods.
  arXiv  Detail & Related papers  (2026-03-17T21:19:08Z)
• RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training [59.493415006017635]
  Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training.<n>Current evaluation relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs.<n>We propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training.
  arXiv  Detail & Related papers  (2026-02-13T12:56:31Z)
• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains [97.5573252172065]
  We train a family of Automatic Reasoning Evaluators (FARE) with a simple iterative rejection-sampling supervised finetuning approach.<n>FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators.<n>As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH.
  arXiv  Detail & Related papers  (2025-10-20T17:52:06Z)
• MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization [103.74675519953898]
  Long-chain reflective reasoning is a prerequisite for solving complex real-world problems.<n>We build a benchmark consisting 1,260 samples of 42 challenging synthetic tasks.<n>We generate post-training data and explore learning paradigms for exploiting such data.
  arXiv  Detail & Related papers  (2025-10-09T17:53:58Z)
• LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model [99.71684530652942]
  We show that LLaVA-Critic-R1 emerges as a top-performing critic but also as a competitive policy model.<n>Applying self-critique at test time yields an average +13.8% improvement on five representative reasoning tasks.<n>Our results reveal that RL training on critic data can produce a unified model excelling at both evaluation and generation.
  arXiv  Detail & Related papers  (2025-08-31T03:08:02Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.