Hybrid Instructor Ai Assessment In Academic Projects: Efficiency, Equity, And Methodological Lessons

• URL: http://arxiv.org/abs/2510.22286v1
• Date: Sat, 25 Oct 2025 13:11:15 GMT
• Title: Hybrid Instructor Ai Assessment In Academic Projects: Efficiency, Equity, And Methodological Lessons
• Abstract summary: This study presents the implementation of a generative artificial intelligence (AI) assisted assessment system, supervised by instructors, to grade 33 hydraulics reports.<n>The quantitative results revealed a noteworthy 88% reduction in grading time and a 733% increase in productivity.<n>The system proved to be equitable, exhibiting no bias related to report length, and highly reliable post-calibration.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In technical subjects characterized by high enrollment, such as Basic Hydraulics, the assessment of reports necessitates superior levels of objectivity, consistency, and formative feedback; goals often compromised by faculty workload. This study presents the implementation of a generative artificial intelligence (AI) assisted assessment system, supervised by instructors, to grade 33 hydraulics reports. The central objective was to quantify its impact on the efficiency, quality, and fairness of the process. The employed methodology included the calibration of the Large Language Model (LLM) with a detailed rubric, the batch processing of assignments, and a human-in-the-loop validation phase. The quantitative results revealed a noteworthy 88% reduction in grading time (from 50 to 6 minutes per report, including verification) and a 733% increase in productivity. The quality of feedback was substantially improved, evidenced by 100% rubric coverage and a 150% increase in the anchoring of comments to textual evidence. The system proved to be equitable, exhibiting no bias related to report length, and highly reliable post-calibration (r = 0.96 between scores). It is concluded that the hybrid AI-instructor model optimizes the assessment process, thereby liberating time for high-value pedagogical tasks and enhancing the fairness and quality of feedback, in alignment with UNESCO's principles on the ethical use of AI in education.

Related papers

• A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment [0.0]
  This study presents the design and validation of an AI-assisted scoring framework for written responses in a large-scale national assessment.<n>The proposed approach focuses on short written texts of approximately 150-200 words and incorporates a human-in-the-loop strategy to preserve assessment quality while reducing manual workload.
  arXiv  Detail & Related papers  (2026-09-04T13:47:29Z)
• An LLM -Powered Assessment Retrieval-Augmented Generation (RAG) For Higher Education [0.0]
  This study presents an agentic assessment system built on a Retrieval-Augmented Generation architecture.<n>The system integrates a large language model with a structured retrieval mechanism that accesses rubric criteria, exemplar essays, and instructor feedback.<n>Results demonstrate that the RAG system can produce reliable, rubric-aligned feedback at scale, achieving 94--99% agreement with human evaluators.
  arXiv  Detail & Related papers  (2026-01-05T15:13:00Z)
• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains [97.5573252172065]
  We train a family of Automatic Reasoning Evaluators (FARE) with a simple iterative rejection-sampling supervised finetuning approach.<n>FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators.<n>As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH.
  arXiv  Detail & Related papers  (2025-10-20T17:52:06Z)
• NAIPv2: Debiased Pairwise Learning for Efficient Paper Quality Estimation [58.30936615525824]
  We present NAIPv2, a debiased and efficient framework for paper quality estimation.<n> NAIPv2 employs pairwise learning within domain-year groups to reduce inconsistencies in reviewer ratings.<n>It is trained on pairwise comparisons but enabling efficient pointwise prediction at deployment.
  arXiv  Detail & Related papers  (2025-09-29T17:59:23Z)
• Teaching at Scale: Leveraging AI to Evaluate and Elevate Engineering Education [3.557803321422781]
  This article presents a scalable, AI-supported framework for qualitative student feedback using large language models.<n>The system employs hierarchical summarization, anonymization, and exception handling to extract actionable themes from open-ended comments.<n>We report on its successful deployment across a large college of engineering.
  arXiv  Detail & Related papers  (2025-08-01T20:27:40Z)
• Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors [6.891852148875869]
  The task consisted of five tracks designed to automatically evaluate the AI tutor's performance across key dimensions of mistake identification.<n>The best results for the four pedagogical ability assessment tracks range between macro F1 scores of 58.34 (for providing guidance) and 71.81 (for mistake identification) on three-class problems.
  arXiv  Detail & Related papers  (2025-07-11T10:57:36Z)
• Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning [63.531262595858]
  Divide-and-conquer approach breaks comprehensive evaluation task into localized scoring tasks, followed by a final global assessment.<n>We introduce a hybrid in-context learning approach that leverages human annotations to enhance the performance of both local and global evaluations.<n>Finally, we develop an uncertainty-based active learning algorithm that efficiently selects data samples for human annotation.
  arXiv  Detail & Related papers  (2025-05-26T16:39:41Z)
• Evaluating the Performance of Nigerian Lecturers using Multilayer Perceptron [0.0]
  The system was designed using a web-based platform, created a secure database, and by using a custom dataset.<n>The model's performance was evaluated using Mean Squared Error (MSE) and Mean Absolute Error (MAE)<n>The model also demonstrated an estimated accuracy rate of approximately 96%, validated its effectiveness in predicting lecturer performance.
  arXiv  Detail & Related papers  (2025-05-22T07:23:14Z)
• J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning [54.85131761693927]
  We introduce J1, a reinforcement learning framework for teaching LLM judges to think before making decisions.<n>Our core contribution lies in converting all judgment tasks for non-verifiable and verifiable prompts into a unified format with verifiable rewards.<n>We then use RL to train thinking-judges at scales of 8B, 32B, and 70B and show that they obtain state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-05-15T14:05:15Z)
• A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher Education [0.6141800972050401]
  We propose a Zero-Shot Large Language Model (LLM)-Based Automated Assignment Grading (AAG) system.<n>This framework leverages prompt engineering to evaluate both computational and explanatory student responses without requiring additional training or fine-tuning.<n>The AAG system delivers tailored feedback that highlights individual strengths and areas for improvement, thereby enhancing student learning outcomes.
  arXiv  Detail & Related papers  (2025-01-24T08:01:41Z)
• A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look [52.114284476700874]
  This paper reports on the results of a large-scale evaluation (the TREC 2024 RAG Track) where four different relevance assessment approaches were deployed. We find that automatically generated UMBRELA judgments can replace fully manual judgments to accurately capture run-level effectiveness. Surprisingly, we find that LLM assistance does not appear to increase correlation with fully manual assessments, suggesting that costs associated with human-in-the-loop processes do not bring obvious tangible benefits.
  arXiv  Detail & Related papers  (2024-11-13T01:12:35Z)
• "I understand why I got this grade": Automatic Short Answer Grading with Feedback [33.63970664152288]
  We introduce Engineering Short Answer Feedback (EngSAF), a dataset designed for automatic short-answer grading with feedback.<n>We incorporate feedback into our dataset by leveraging the generative capabilities of state-of-the-art large language models (LLMs) using our Label-Aware Synthetic Feedback Generation (LASFG) strategy.<n>The best-performing model (Mistral-7B) achieves an overall accuracy of 75.4% and 58.7% on unseen answers and unseen question test sets, respectively.
  arXiv  Detail & Related papers  (2024-06-30T15:42:18Z)
• Evaluating Mathematical Reasoning Beyond Accuracy [50.09931172314218]
  We introduce ReasonEval, a new methodology for evaluating the quality of reasoning steps.<n>We show that ReasonEval consistently outperforms baseline methods in the meta-evaluation datasets.<n>We observe that ReasonEval can play a significant role in data selection.
  arXiv  Detail & Related papers  (2024-04-08T17:18:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.