Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards

• URL: http://arxiv.org/abs/2607.19219v1
• Date: Tue, 21 Jul 2026 15:49:02 GMT
• Title: Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards
• Abstract summary: Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG)<n>We propose RLAES, a unified LLM framework that jointly optimize essay scoring and feedback generation through RL.<n>To make feedback quality measurable, interpretable, and usable for training, we introduce RL-based Feedback Evaluation (RFE)
• Score: 6.002323709229882
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine-tuning, while systematic research on reinforcement learning (RL) post-training and automated evaluation of feedback quality remains limited. We propose RLAES, a unified LLM framework that jointly optimizes essay scoring and feedback generation through RL. To make feedback quality measurable, interpretable, and usable for training, we introduce Rubric-based Feedback Evaluation (RFE), an essay-grounded feedback evaluation framework comprising 166 fine-grained binary rubric items and an LLM-as-judge. Building on RFE, we propose Adaptive Gated Feedback Optimization (AGFO), which activates rubric-based feedback rewards on demand during RL, reducing evaluation overhead while improving feedback quality. We also propose Adjacent Contrastive Reasoning (ACR) to improve ordinal score calibration by explicitly contrasting adjacent score levels. Experimental results show that the RFE framework captures essay-feedback consistency, exhibits strong pairwise discriminative power, and closely aligns with expert preferences. On the ASAP benchmark, RLAES-AGFO achieves the best scoring performance among LLM-based methods (QWK = 0.803), while maintaining feedback quality comparable to GPT-5.5 and avoiding the feedback degradation observed under score-only RL. Code and datasets are publicly available at https://github.com/hellomuyi/RLAES.

Related papers

• Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation [50.696688705287755]
  We propose Mutual Information Self-Evaluation to overcome the sparse reward challenge in reinforcement learning.<n>MISE enables an agent to learn autonomously from dense internal rewards supplementing sparse extrinsic signals.<n>We prove that utilizing hindsight self-evaluation rewards is equivalent to minimizing an objective that combines mutual information with a KL divergence term between the policy and a proxy reward policy.
  arXiv  Detail & Related papers  (2026-04-13T15:18:51Z)
• Evaluating LLM-Based Grant Proposal Review via Structured Perturbations [18.689211845609623]
  We develop a perturbation-based framework probing LLM sensitivity across six quality axes.<n>We compare three review architectures: single-pass review, section-by-section analysis, and a 'Council of Personas' ensemble emulating expert panels.
  arXiv  Detail & Related papers  (2026-03-09T11:53:50Z)
• FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback [6.88204255655161]
  We propose FeedEval, a framework for evaluating LLM-generated essay feedback along three pedagogically grounded dimensions: specificity, helpfulness, and validity.<n>Experiments on the ASAP++ benchmark show that FeedEval closely aligns with human expert judgments and that essay scoring models trained with FeedEval-filtered high-quality feedback achieve superior scoring performance.
  arXiv  Detail & Related papers  (2026-01-08T04:04:29Z)
• IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation [87.38454788767545]
  We propose IF-CRITIC, an evaluation model for instruction following in Large Language Models.<n>With the scalable reward signals provided by IF-CRITIC, LLMs can achieve substantial performance gains in instruction-following optimization.
  arXiv  Detail & Related papers  (2025-11-02T17:06:49Z)
• Benchmarking and Studying the LLM-based Code Review [34.93646390349726]
  Current benchmarks frequently focus on fine-grained code units, lack complete project context, and use inadequate evaluation metrics.<n>We introduce SWRBench, a new benchmark offering PR-centric review with full project context.<n>Our contributions include the SWRBench benchmark, its objective evaluation method, a comprehensive study of current ACR capabilities, and an effective enhancement approach.
  arXiv  Detail & Related papers  (2025-09-01T14:13:34Z)
• Training Language Model to Critique for Better Refinement [58.73039433159486]
  We introduce textbfRefinement-oriented textbfCritique textbfOptimization (RCO), a novel framework designed to train critic models using refinement signals.<n>RCO uses a feedback loop where critiques, generated by the critic model, guide the actor model in refining its responses.<n>By focusing on critiques that lead to better refinements, RCO eliminates the need for direct critique preference assessment.
  arXiv  Detail & Related papers  (2025-06-27T12:10:57Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)
• Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation [57.380464382910375]
  We show that the choice of feedback protocol for evaluation can significantly affect evaluation reliability and induce systematic biases.<n>We find that generator models can flip preferences by embedding distractor features.<n>We offer recommendations for choosing feedback protocols based on dataset characteristics and evaluation objectives.
  arXiv  Detail & Related papers  (2025-04-20T19:05:59Z)
• Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators [48.54465599914978]
  Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language.<n>LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments.<n>We introduce Pairwise-preference Search (PAIRS), an uncertainty-guided search-based rank aggregation method that employs LLMs to conduct pairwise comparisons locally and efficiently ranks candidate texts globally.
  arXiv  Detail & Related papers  (2024-03-25T17:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.