Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring

• URL: http://arxiv.org/abs/2605.25731v2
• Date: Tue, 26 May 2026 10:44:35 GMT
• Title: Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring
• Abstract summary: Trait-Aware Policy Optimization (TAPO) is a post-training framework tailored to autoregressive multi-trait scoring.<n>Our method decomposes rewards along both the sample and trait dimensions, combining global scoring consistency, trait-level accuracy, format validity, and inter-trait dependency preservation.
• Score: 27.596544379317436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-trait essay scoring aims to provide fine-grained evaluation of writing quality across multiple dimensions. However, how to effectively post-train autoregressive scoring models remains underexplored. In this paper, we propose Trait-Aware Policy Optimization (TAPO), a post-training framework tailored to autoregressive multi-trait scoring. Our method decomposes rewards along both the sample and trait dimensions, combining global scoring consistency, trait-level accuracy, format validity, and inter-trait dependency preservation. In addition, we use enhanced prompts throughout training by incorporating original prompt texts and trait descriptions, providing richer semantic information for trait-specific score generation. Experiments across multiple backbone models show that our method consistently improves multi-trait scoring performance over supervised fine-tuning and scalar-reward optimization baselines, demonstrating the effectiveness and transferability of trait-aware post-training for essay scoring.

Related papers

• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• MASF: A Multi-Model Adaptive Selection Framework for Abstractive Text summarization [0.2864713389096699]
  This paper presents a Multi-Model Adaptive Summarization Framework to improve the robustness and quality of abstractive text summarization.<n>It integrates multiple fine-tuned transformer-based summarization models and introduces an adaptive selection mechanism.<n>It achieves the highest BERTScore among all compared methods with a score of 88.63%.
  arXiv  Detail & Related papers  (2026-06-03T22:34:40Z)
• Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics [0.9867902290081937]
  We introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics.<n>Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics.<n>Experiments demonstrate consistent factuality gains across models, ranging from early encoder-decoder architectures to modern large language models.
  arXiv  Detail & Related papers  (2026-05-26T10:55:03Z)
• Threshold-Guided Optimization for Visual Generative Models [16.556017381410943]
  We propose a threshold-guided framework for aligning visual generative models with human feedback.<n>Our method consistently improves preference alignment over previous methods.<n>These results position our threshold-guided framework as a simple yet principled alternative for aligning visual generative models without paired comparisons.
  arXiv  Detail & Related papers  (2026-05-06T08:59:16Z)
• Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models [54.597200388738656]
  Reward-based fine-tuning aims to steer a pretrained diffusion or flow-based generative model toward higher-reward samples.<n>We show that many can be written under a common framework, which we call reward score matching (RSM)
  arXiv  Detail & Related papers  (2026-04-19T12:47:52Z)
• Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization [5.909152930998754]
  Multi-role dialogue summarization requires modeling complex interactions among multiple speakers.<n>Most existing methods optimize for automatic metrics such as ROUGE and BERTScore.<n>We propose a novel framework that couples explicit cognitive-style reasoning with reward-based optimization.
  arXiv  Detail & Related papers  (2026-04-19T01:27:41Z)
• Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models [63.00458229517523]
  This work addresses the evaluation challenge of reward models by probing preference representations.<n>We construct a Multi-dimensional Reward Model Benchmark (MRMBench), a collection of six probing tasks for different preference dimensions.<n>We introduce an analysis method, inference-time probing, which identifies the dimensions used during the reward prediction and enhances its interpretability.
  arXiv  Detail & Related papers  (2025-11-16T05:29:29Z)
• Multi-Metric Preference Alignment for Generative Speech Restoration [15.696247605348383]
  We propose a multi-metric preference alignment strategy for generative models.<n>We observe consistent and significant performance gains across three diverse generative paradigms.<n>Our aligned models can serve as powerful ''data annotators'', generating high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2025-08-24T07:05:10Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Learning Dynamic Representations via An Optimally-Weighted Maximum Mean Discrepancy Optimization Framework for Continual Learning [16.10753846850319]
  Continual learning allows models to persistently acquire and retain information.<n> catastrophic forgetting can severely impair model performance.<n>We introduce a novel framework termed Optimally-Weighted Mean Discrepancy (OWMMD), which imposes penalties on representation alterations.
  arXiv  Detail & Related papers  (2025-01-21T13:33:45Z)
• Hateful Meme Detection through Context-Sensitive Prompting and Fine-Grained Labeling [9.166963162285064]
  We propose an end-to-end conceptual framework for model optimization in complex tasks. Experiments support the efficacy of this traditional yet novel framework, achieving the highest accuracy and AUROC.
  arXiv  Detail & Related papers  (2024-11-13T08:05:41Z)
• Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models [85.96013373385057]
  Fine-tuning text-to-image models with reward functions trained on human feedback data has proven effective for aligning model behavior with human intent. However, excessive optimization with such reward models, which serve as mere proxy objectives, can compromise the performance of fine-tuned models. We propose TextNorm, a method that enhances alignment based on a measure of reward model confidence estimated across a set of semantically contrastive text prompts.
  arXiv  Detail & Related papers  (2024-04-02T11:40:38Z)
• Enhancing Coherence of Extractive Summarization with Multitask Learning [40.349019691412465]
  This study proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and coherent discriminator module. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in the extracted summaries.
  arXiv  Detail & Related papers  (2023-05-22T09:20:58Z)
• $\ extit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation [65.29170569821093]
  parallel text generation has received widespread attention due to its success in generation efficiency. In this paper, we propose $textitlatent$-GLAT, which employs the discrete latent variables to capture word categorical information. Experiment results show that our method outperforms strong baselines without the help of an autoregressive model.
  arXiv  Detail & Related papers  (2022-04-05T07:34:12Z)
• Text Counterfactuals via Latent Optimization and Shapley-Guided Search [15.919650185010491]
  We study the problem of generating counterfactual text for a classification model. We aim to minimally alter the text to change the model's prediction. White-box approaches have been successfully applied to similar problems in vision.
  arXiv  Detail & Related papers  (2021-10-22T05:04:40Z)
• PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies [68.8204255655161]
  We propose a reinforcement learning based framework PoBRL for solving multi-document summarization. Our strategy decouples this multi-objective optimization into different subproblems that can be solved individually by reinforcement learning. Our empirical analysis shows state-of-the-art performance on several multi-document datasets.
  arXiv  Detail & Related papers  (2021-05-18T02:55:42Z)
• On Learning Text Style Transfer with Direct Rewards [101.97136885111037]
  Lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task. We leverage semantic similarity metrics originally used for fine-tuning neural machine translation models. Our model provides significant gains in both automatic and human evaluation over strong baselines.
  arXiv  Detail & Related papers  (2020-10-24T04:30:02Z)
• Automated Concatenation of Embeddings for Structured Prediction [75.44925576268052]
  We propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model.
  arXiv  Detail & Related papers  (2020-10-10T14:03:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.