APTER: Adaptive Post-Training with Expert-Grounded Rubrics

• URL: http://arxiv.org/abs/2608.14212v1
• Date: Fri, 14 Aug 2026 11:42:29 GMT
• Title: APTER: Adaptive Post-Training with Expert-Grounded Rubrics
• Abstract summary: APTER is a framework that integrates structured domain knowledge into fine-grained evaluation, optimization, and diagnosis for specialized complex reasoning.<n>APTER improves the mathematics and medical averages over the corresponding base models by up to 15.86 and 8.04 points, respectively.
• Score: 23.84845758379717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As large language models enter professional domains, they must satisfy domain constraints, include critical evidence, and provide complete reasoning rather than merely produce fluent responses. Existing post-training methods often rely on holistic preferences or outcome-level verification, while recent rubric-based methods usually generate rubrics independently for each query. In specialized domains, such unconstrained rubrics may omit critical requirements and vary across samples, hindering the diagnosis and targeted repair of persistent capability deficiencies. We propose APTER (Adaptive Post-Training with Expert-Grounded Rubrics), a framework that integrates structured domain knowledge into fine-grained evaluation, optimization, and diagnosis for specialized complex reasoning. First, expert-grounded rubric construction starts from an expert criteria framework built by domain experts, where each criterion represents a stable professional capability. For each query, APTER selects relevant criteria and instantiates them into query-level rubrics linked to their source criteria, turning reusable expert criteria into executable query-level supervision without reference answers. Second, adaptive post-training uses rubric verdicts as both optimization and criterion-level diagnostic signals. Aggregating low-scoring verdicts by criterion ID reveals persistent deficiencies and triggers targeted supervised fine-tuning updates during reinforcement learning. Experiments on mathematical reasoning and medical question answering show consistent gains across both domains. Across three model generations, APTER improves the mathematics and medical averages over the corresponding base models by up to 15.86 and 8.04 points, respectively. Code and rubric datasets are available at https://github.com/AntDT-APTER/APTER.

Related papers

• GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation [74.44235943118994]
  GenRubric is a self-evolving framework that improves rubric generation from unlabeled queries.<n>We implement this principle through reinforcement learning, combining a cross-rubric comprehensiveness signal with group-level and criterion-level rewards for rubric quality.<n> Experiments on human-annotated rubric benchmarks show that self-evolution improves the agreement between evaluations induced by generated rubrics and those induced by expert-written rubrics.
  arXiv  Detail & Related papers  (2026-08-30T15:39:39Z)
• Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval [0.0]
  Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding their responses in external knowledge.<n>This paper presents an Agent-Orchestrated Adaptive RAG framework that introduces dynamic query decomposition, iterative retrieval, and a bounded self-reflective evaluation loop.
  arXiv  Detail & Related papers  (2026-06-04T03:38:46Z)
• CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging [95.02210956333374]
  We propose a criterion-centric framework that adapts rubric-based judging to pairwise preference evaluation.<n>On BigCodeReward, CriterAlign improves a Qwen2.5-VL-32B monolithic judge from 60.4% to 66.3% accuracy.
  arXiv  Detail & Related papers  (2026-05-19T10:59:19Z)
• DepthCharge: A Domain-Agnostic Framework for Measuring Depth-Dependent Knowledge in Large Language Models [51.56484100374058]
  Large Language Models appear competent when answering general questions but often fail when pushed into domain-specific details.<n>We present DepthCharge, a domain-agnostic framework that measures knowledge depth through three innovations.<n> adaptive probing that generates follow-up questions based on concepts the model actually mentions, on-demand fact verification from authoritative sources, and survival statistics with constant sample sizes at every depth level.
  arXiv  Detail & Related papers  (2026-03-05T20:49:11Z)
• Automated Benchmark Generation from Domain Guidelines Informed by Bloom's Taxonomy [28.293009223912602]
  Open-ended question answering (QA) evaluates a model's ability to perform contextualized reasoning beyond factual recall.<n>This challenge is especially acute in practice-based domains, where knowledge is procedural and grounded in professional judgment.<n>We introduce a framework for automated benchmark generation from expert-authored guidelines informed by Bloom's taxonomy.
  arXiv  Detail & Related papers  (2026-01-28T05:01:11Z)
• RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG [0.0]
  Retrieval-Augmented Generation (RAG) is a critical technique for grounding Large Language Models (LLMs) in factual evidence.<n>Existing evaluation frameworks often rely on metrics that fail to capture domain-specific nuances.<n>This paper introduces RAGalyst, an automated, human-aligned agentic framework designed for the rigorous evaluation of domain-specific RAG systems.
  arXiv  Detail & Related papers  (2025-11-06T16:22:52Z)
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation [55.47971671635531]
  Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA)<n>Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge.<n>Existing systems primarily rely on unstructured documents, while largely overlooking relational databases.
  arXiv  Detail & Related papers  (2025-09-30T22:19:44Z)
• Beyond Overall Accuracy: A Psychometric Deep Dive into the Topic-Specific Medical Capabilities of 80 Large Language Models [6.362188639024662]
  We introduce textscMedIRT, a rigorous evaluation framework grounded in Item Response Theory (IRT)<n>We prospectively gathered fresh responses from 80 diverse Large Language Models (LLMs) on a balanced, 1,100-question USMLE-aligned benchmark.<n>We estimate LLM's latent model ability jointly with question difficulty and discrimination, yielding more stable and nuanced performance rankings than accuracy alone.
  arXiv  Detail & Related papers  (2025-09-29T02:06:13Z)
• EvalAgent: Discovering Implicit Evaluation Criteria from the Web [82.82096383262068]
  We introduce EvalAgent, a framework designed to automatically uncover nuanced and task-specific criteria.<n>EvalAgent mines expert-authored online guidance to propose diverse, long-tail evaluation criteria.<n>Our experiments demonstrate that the grounded criteria produced by EvalAgent are often implicit, yet specific.
  arXiv  Detail & Related papers  (2025-04-21T16:43:50Z)
• Computational Complexity of Preferred Subset Repairs on Data-Graphs [2.254434034390529]
  We study the problem of computing prioritized repairs over graph databases with data values. We present several preference criteria based on the standard subset repair semantics. We show that it is possible to maintain the same computational complexity as in the case where no preference criterion is available for exploitation.
  arXiv  Detail & Related papers  (2024-02-14T15:51:55Z)
• Generalizable Person Re-Identification via Self-Supervised Batch Norm Test-Time Adaption [63.7424680360004]
  Batch Norm Test-time Adaption (BNTA) is a novel re-id framework that applies the self-supervised strategy to update BN parameters adaptively. BNTA explores the domain-aware information within unlabeled target data before inference, and accordingly modulates the feature distribution normalized by BN to adapt to the target domain.
  arXiv  Detail & Related papers  (2022-03-01T18:46:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.