TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring

• URL: http://arxiv.org/abs/2505.14577v2
• Date: Sat, 31 May 2025 11:33:14 GMT
• Title: TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring
• Authors: Sohaila Eltanbouly, Salam Albatarni, Tamer Elsayed,
• Abstract summary: TRATES is a novel trait-specific and rubric-based cross-prompt AES framework that is generic yet specific to the underlying trait.<n>The framework leverages a Large Language Model (LLM) that utilizes the trait grading rubrics to generate trait-specific features.<n>Experiments show that TRATES achieves a new state-of-the-art performance across all traits on a widely-used dataset.
• Score: 4.021352247826289
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Research on holistic Automated Essay Scoring (AES) is long-dated; yet, there is a notable lack of attention for assessing essays according to individual traits. In this work, we propose TRATES, a novel trait-specific and rubric-based cross-prompt AES framework that is generic yet specific to the underlying trait. The framework leverages a Large Language Model (LLM) that utilizes the trait grading rubrics to generate trait-specific features (represented by assessment questions), then assesses those features given an essay. The trait-specific features are eventually combined with generic writing-quality and prompt-specific features to train a simple classical regression model that predicts trait scores of essays from an unseen prompt. Experiments show that TRATES achieves a new state-of-the-art performance across all traits on a widely-used dataset, with the generated LLM-based features being the most significant.

Related papers

• LATex: Leveraging Attribute-based Text Knowledge for Aerial-Ground Person Re-Identification [63.07563443280147]
  We propose a novel framework named LATex for AG-ReID.<n>It adopts prompt-tuning strategies to leverage attribute-based text knowledge.<n>Our framework can fully leverage attribute-based text knowledge to improve the AG-ReID.
  arXiv  Detail & Related papers  (2025-03-31T04:47:05Z)
• Improving RAG for Personalization with Author Features and Contrastive Examples [2.6968321526169503]
  Personalization with retrieval-augmented generation (RAG) often fails to capture fine-grained features of authors.<n>We introduce Contrastive Examples: documents from other authors are retrieved to help LLM identify what makes an author's style unique in comparison to others.<n>Our results show the value of fine-grained features for better personalization, while opening a new research dimension for including contrastive examples as a complement with RAG.
  arXiv  Detail & Related papers  (2025-03-24T01:41:22Z)
• Multi-field Visualization: Trait design and trait-induced merge trees [2.862576303934634]
  Feature level sets (FLS) have shown significant potential in the analysis of multi-field data by using traits defined in attribute space to specify features.<n>In this work, we address key challenges in the practical use of FLS: trait design and feature selection for rendering.<n>We propose a decomposition of traits into simpler components, making the process more intuitive and computationally efficient.
  arXiv  Detail & Related papers  (2025-01-08T10:13:32Z)
• Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs [2.324913904215885]
  This paper introduces Rationale-based Multiple Trait Scoring (RMTS), a novel approach for multi-trait essay scoring.<n>RMTS integrates prompt-engineering-based large language models (LLMs) with a fine-tuning-based essay scoring model using a smaller large language model (S-LLM)<n>Experiments on benchmark datasets, including ASAP, ASAP++, and Feedback Prize, show that RMTS significantly outperforms state-of-the-art models and vanilla S-LLMs in trait-specific scoring.
  arXiv  Detail & Related papers  (2024-10-18T06:35:17Z)
• BookWorm: A Dataset for Character Description and Analysis [59.186325346763184]
  We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation. We introduce the BookWorm dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks.
  arXiv  Detail & Related papers  (2024-10-14T10:55:58Z)
• PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models [3.516029765200171]
  We propose a high-quality, personalized, manually annotated abstractive summarization dataset called PersonalSum. This dataset is the first to investigate whether the focus of public readers differs from the generic summaries generated by Large Language Models.
  arXiv  Detail & Related papers  (2024-10-04T20:12:39Z)
• Editing Personality for Large Language Models [73.59001811199823]
  This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs) We construct PersonalityEdit, a new benchmark dataset to address this task.
  arXiv  Detail & Related papers  (2023-10-03T16:02:36Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoring [3.6825890616838066]
  Automated essay scoring (AES) aims to score essays written for a given prompt, which defines the writing topic. Most existing AES systems assume to grade essays of the same prompt as used in training and assign only a holistic score. We propose a robust model: prompt- and trait relation-aware cross-prompt essay trait scorer.
  arXiv  Detail & Related papers  (2023-05-26T11:11:19Z)
• An Interpretable End-to-end Fine-tuning Approach for Long Clinical Text [72.62848911347466]
  Unstructured clinical text in EHRs contains crucial information for applications including decision support, trial matching, and retrospective research. Recent work has applied BERT-based models to clinical information extraction and text classification, given these models' state-of-the-art performance in other NLP domains. In this work, we propose a novel fine-tuning approach called SnipBERT. Instead of using entire notes, SnipBERT identifies crucial snippets and feeds them into a truncated BERT-based model in a hierarchical manner.
  arXiv  Detail & Related papers  (2020-11-12T17:14:32Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling [81.33107307509718]
  We propose a topic adaptive storyteller to model the ability of inter-topic generalization. We also propose a prototype encoding structure to model the ability of intra-topic derivation. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model.
  arXiv  Detail & Related papers  (2020-08-11T03:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.