RubiSCoT: A Framework for AI-Supported Academic Assessment

• URL: http://arxiv.org/abs/2510.17309v1
• Date: Mon, 20 Oct 2025 08:52:33 GMT
• Title: RubiSCoT: A Framework for AI-Supported Academic Assessment
• Authors: Thorsten Fröhlich, Tim Schlippe,
• Abstract summary: RubiSCoT is an AI-supported framework designed to enhance thesis evaluation from proposal to final submission.<n>The framework includes preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting.
• Score: 0.042970700836450486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The evaluation of academic theses is a cornerstone of higher education, ensuring rigor and integrity. Traditional methods, though effective, are time-consuming and subject to evaluator variability. This paper presents RubiSCoT, an AI-supported framework designed to enhance thesis evaluation from proposal to final submission. Using advanced natural language processing techniques, including large language models, retrieval-augmented generation, and structured chain-of-thought prompting, RubiSCoT offers a consistent, scalable solution. The framework includes preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting. We present the design and implementation of RubiSCoT, discussing its potential to optimize academic assessment processes through consistent, scalable, and transparent evaluation.

Related papers

• Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark [9.922581736690159]
  We present a large-scale empirical study of AI grading on real, handwritten calculus work from UC Irvine.<n>Using OCR-conditioned large language models with structured, rubric-guided prompting, our system produces scores and formative feedback for thousands of free-response quiz submissions.<n>In a setting with no single ground-truth label, we evaluate performance against official teaching-assistant grades, student surveys, and independent human review.
  arXiv  Detail & Related papers  (2026-03-01T03:32:51Z)
• Teaching at Scale: Leveraging AI to Evaluate and Elevate Engineering Education [3.557803321422781]
  This article presents a scalable, AI-supported framework for qualitative student feedback using large language models.<n>The system employs hierarchical summarization, anonymization, and exception handling to extract actionable themes from open-ended comments.<n>We report on its successful deployment across a large college of engineering.
  arXiv  Detail & Related papers  (2025-08-01T20:27:40Z)
• Ratas framework: A comprehensive genai-based approach to rubric-based marking of real-world textual exams [3.4132239125074206]
  RATAS (Rubric Automated Tree-based Answer Scoring) is a novel framework that leverages state-of-the-art generative AI models for rubric-based grading of textual responses.<n> RATAS is designed to support a wide range of grading rubrics, enable subject-agnostic evaluation, and generate structured, explainable rationales for assigned scores.
  arXiv  Detail & Related papers  (2025-05-27T22:17:27Z)
• Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning [63.531262595858]
  Divide-and-conquer approach breaks comprehensive evaluation task into localized scoring tasks, followed by a final global assessment.<n>We introduce a hybrid in-context learning approach that leverages human annotations to enhance the performance of both local and global evaluations.<n>Finally, we develop an uncertainty-based active learning algorithm that efficiently selects data samples for human annotation.
  arXiv  Detail & Related papers  (2025-05-26T16:39:41Z)
• Measurement to Meaning: A Validity-Centered Framework for AI Evaluation [12.55408229639344]
  We provide a structured approach for reasoning about the types of evaluative claims that can be made given the available evidence.<n>Our framework is well-suited for the contemporary paradigm in machine learning.
  arXiv  Detail & Related papers  (2025-05-13T20:36:22Z)
• Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework [61.38174427966444]
  Large Language Models (LLMs) are being used more and more extensively for automated evaluation in various scenarios.<n>Previous studies have attempted to fine-tune open-source LLMs to replicate the evaluation explanations and judgments of powerful proprietary models.<n>We propose a novel evaluation framework, ARJudge, that adaptively formulates evaluation criteria and synthesizes both text-based and code-driven analyses.
  arXiv  Detail & Related papers  (2025-02-26T06:31:45Z)
• ReLearn: Unlearning via Learning for Large Language Models [64.2802606302194]
  We propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning.<n>This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation.<n>Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output.
  arXiv  Detail & Related papers  (2025-02-16T16:31:00Z)
• StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs [78.84060166851805]
  StructTest is a novel benchmark that evaluates large language models (LLMs) on their ability to follow compositional instructions and generate structured outputs.<n> Assessments are conducted deterministically using a rule-based evaluator, which can be easily extended to new tasks and datasets.<n>We demonstrate that StructTest remains challenging even for top-performing models like Deepseek-V3/R1 and GPT-4o.
  arXiv  Detail & Related papers  (2024-12-23T22:08:40Z)
• Improving Academic Skills Assessment with NLP and Ensemble Learning [7.803554057024728]
  This study addresses the critical challenges of assessing foundational academic skills by leveraging advancements in natural language processing (NLP) Our approach integrates multiple state-of-the-art NLP models, including BERT, RoBERTa, BART, DeBERTa, and T5. The methodology involves detailed data preprocessing, feature extraction, and pseudo-label learning to optimize model performance.
  arXiv  Detail & Related papers  (2024-09-23T23:43:43Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Multi-Dimensional Evaluation of Text Summarization with In-Context Learning [79.02280189976562]
  In this paper, we study the efficacy of large language models as multi-dimensional evaluators using in-context learning. Our experiments show that in-context learning-based evaluators are competitive with learned evaluation frameworks for the task of text summarization. We then analyze the effects of factors such as the selection and number of in-context examples on performance.
  arXiv  Detail & Related papers  (2023-06-01T23:27:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.