Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis

• URL: http://arxiv.org/abs/2511.18221v1
• Date: Sat, 22 Nov 2025 23:43:00 GMT
• Title: Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis
• Abstract summary: We enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints.<n>The correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data.
• Score: 5.995976425903575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research full paper presents an enhancement pipeline for large language models (LLMs) in assessing homework for an undergraduate circuit analysis course, aiming to improve LLMs' capacity to provide personalized support to electrical engineering students. Existing evaluations have demonstrated that GPT-4o possesses promising capabilities in assessing student homework in this domain. Building on these findings, we enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints. These strategies effectively address common errors observed in GPT-4o's responses when using simple prompts, leading to a substantial improvement in assessment accuracy. Specifically, the correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data on entry-level circuit analysis topics. This work lays a foundation for the effective integration of LLMs into circuit analysis instruction and, more broadly, into engineering education.

Related papers

• Evaluating Accounting Reasoning Capabilities of Large Language Models [21.704851858354157]
  We propose evaluation criteria derived from an analysis of the training data characteristics of representative GLM models.<n>We evaluate GLM-6B, GLM-130B, GLM-4, and OpenAI GPT-4 on accounting reasoning tasks.<n>Results show that prompt design significantly affects performance, with GPT-4 demonstrating the strongest capability.
  arXiv  Detail & Related papers  (2026-01-10T22:24:52Z)
• Automated Refinement of Essay Scoring Rubrics for Language Models via Reflect-and-Revise [41.16092952642748]
  This study investigates the potential for enhancing Automated Scoring (AES) by refining rubrics used by Large Language Models (LLMs)<n> Experiments on datasets using GPT-4.1, Gemini-2.5-Pro, and Qwen-3-Next-80B-A3B-Instruct show Quadratic Weighted Kappa (QWK) improvements of up to 0.19 and 0.47, respectively.
  arXiv  Detail & Related papers  (2025-10-10T06:05:38Z)
• Benchmarking Large Language Models on Homework Assessment in Circuit Analysis [5.040027178589699]
  Large language models (LLMs) have the potential to revolutionize various fields, including code development, robotics, finance, and education.<n>This paper investigates how LLMs can be leveraged in engineering education.<n>We benchmark the capabilities of different LLMs, including GPT-3.5 Turbo, GPT-4o, and Llama 3 70B.
  arXiv  Detail & Related papers  (2025-06-05T15:16:30Z)
• REARANK: Reasoning Re-ranking Agent via Reinforcement Learning [69.8397511935806]
  We present REARANK, a large language model (LLM)-based listwise reasoning reranking agent.<n>REARANK explicitly reasons before reranking, significantly improving both performance and interpretability.
  arXiv  Detail & Related papers  (2025-05-26T14:31:48Z)
• Empirical evaluation of LLMs in predicting fixes of Configuration bugs in Smart Home System [0.0]
  This study evaluates the effectiveness of Large Language Models (LLMs) in predicting fixes for configuration bugs in smart home systems.<n>The research analyzes three prominent LLMs - GPT-4, GPT-4o (GPT-4 Turbo), and Claude 3.5 Sonnet.
  arXiv  Detail & Related papers  (2025-02-16T02:11:36Z)
• CIRCUIT: A Benchmark for Circuit Interpretation and Reasoning Capabilities of LLMs [15.34624510334892]
  The role of Large Language Models (LLMs) has not been extensively explored in analog circuit design.<n>We created the CIRCUIT dataset consisting of 510 question-answer pairs spanning various levels of analog-circuit-related subjects.<n>The best-performing model on our dataset, GPT-4o, achieves 48.04% accuracy when evaluated on the final numerical answer.
  arXiv  Detail & Related papers  (2025-02-11T21:53:48Z)
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants [176.39275404745098]
  We evaluate whether two AI assistants, GPT-3.5 and GPT-4, can adequately answer assessment questions.<n>GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions.<n>Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
  arXiv  Detail & Related papers  (2024-08-07T12:11:49Z)
• InFoBench: Evaluating Instruction Following Ability in Large Language Models [57.27152890085759]
  Decomposed Requirements Following Ratio (DRFR) is a new metric for evaluating Large Language Models' (LLMs) ability to follow instructions. We present InFoBench, a benchmark comprising 500 diverse instructions and 2,250 decomposed questions across multiple constraint categories.
  arXiv  Detail & Related papers  (2024-01-07T23:01:56Z)
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) [121.42924593374127]
  We analyze the latest model, GPT-4V, to deepen the understanding of LMMs. GPT-4V's unprecedented ability in processing arbitrarily interleaved multimodal inputs makes it a powerful multimodal generalist system. GPT-4V's unique capability of understanding visual markers drawn on input images can give rise to new human-computer interaction methods.
  arXiv  Detail & Related papers  (2023-09-29T17:34:51Z)
• Is GPT-4 a Good Data Analyst? [67.35956981748699]
  We consider GPT-4 as a data analyst to perform end-to-end data analysis with databases from a wide range of domains. We design several task-specific evaluation metrics to systematically compare the performance between several professional human data analysts and GPT-4. Experimental results show that GPT-4 can achieve comparable performance to humans.
  arXiv  Detail & Related papers  (2023-05-24T11:26:59Z)
• Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice [0.0]
  This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. The findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems.
  arXiv  Detail & Related papers  (2023-04-20T16:54:34Z)
• GPT-4 Technical Report [116.90398195245983]
  GPT-4 is a large-scale, multimodal model which can accept image and text inputs and produce text outputs. It exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers.
  arXiv  Detail & Related papers  (2023-03-15T17:15:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.