WIP: Large Language Model-Enhanced Smart Tutor for Undergraduate Circuit Analysis

• URL: http://arxiv.org/abs/2506.08962v1
• Date: Tue, 10 Jun 2025 16:35:45 GMT
• Title: WIP: Large Language Model-Enhanced Smart Tutor for Undergraduate Circuit Analysis
• Authors: Liangliang Chen, Huiru Xie, Jacqueline Rohde, Ying Zhang,
• Abstract summary: We detail the tutor's design philosophy and core components, including open-ended question answering and homework feedback generation.<n>The smart tutor was deployed on the Microsoft Azure platform and is currently in use in an undergraduate circuit analysis course.<n>To evaluate its effectiveness, we collected student feedback, with 90.9% of responses indicating satisfaction with the tutor.
• Score: 5.407172767172112
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research-to-practice work-in-progress (WIP) paper presents an AI-enabled smart tutor designed to provide homework assessment and feedback for students in an undergraduate circuit analysis course. We detail the tutor's design philosophy and core components, including open-ended question answering and homework feedback generation. The prompts are carefully crafted to optimize responses across different problems. The smart tutor was deployed on the Microsoft Azure platform and is currently in use in an undergraduate circuit analysis course at the School of Electrical and Computer Engineering in a large, public, research-intensive institution in the Southeastern United States. Beyond offering personalized instruction and feedback, the tutor collects student interaction data, which is summarized and shared with the course instructor. To evaluate its effectiveness, we collected student feedback, with 90.9% of responses indicating satisfaction with the tutor. Additionally, we analyze a subset of collected data on preliminary circuit analysis topics to assess tutor usage frequency for each problem and identify frequently asked questions. These insights help instructors gain real-time awareness of student difficulties, enabling more targeted classroom instruction. In future work, we will release a full analysis once the complete dataset is available after the Spring 2025 semester. We also explore the potential applications of this smart tutor across a broader range of engineering disciplines by developing improved prompts, diagram-recognition methods, and database management strategies, which remain ongoing areas of research.

Related papers

• MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors [76.1634959528817]
  We present MathTutorBench, an open-source benchmark for holistic tutoring model evaluation.<n>MathTutorBench contains datasets and metrics that broadly cover tutor abilities as defined by learning sciences research in dialog-based teaching.<n>We evaluate a wide set of closed- and open-weight models and find that subject expertise, indicated by solving ability, does not immediately translate to good teaching.
  arXiv  Detail & Related papers  (2025-02-26T08:43:47Z)
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants [176.39275404745098]
  We evaluate whether two AI assistants, GPT-3.5 and GPT-4, can adequately answer assessment questions.<n>GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions.<n>Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
  arXiv  Detail & Related papers  (2024-08-07T12:11:49Z)
• "I understand why I got this grade": Automatic Short Answer Grading with Feedback [33.63970664152288]
  We introduce Engineering Short Answer Feedback (EngSAF), a dataset designed for automatic short-answer grading with feedback.<n>We incorporate feedback into our dataset by leveraging the generative capabilities of state-of-the-art large language models (LLMs) using our Label-Aware Synthetic Feedback Generation (LASFG) strategy.<n>The best-performing model (Mistral-7B) achieves an overall accuracy of 75.4% and 58.7% on unseen answers and unseen question test sets, respectively.
  arXiv  Detail & Related papers  (2024-06-30T15:42:18Z)
• The AI Companion in Education: Analyzing the Pedagogical Potential of ChatGPT in Computer Science and Engineering [1.120999712480549]
  This study aims to comprehensively analyze the pedagogical potential of ChatGPT in CSE education. We employ a systematic approach, creating a diverse range of educational practice problems within CSE field. According to our examinations, certain question types, like conceptual knowledge queries, typically do not pose significant challenges to ChatGPT.
  arXiv  Detail & Related papers  (2024-04-23T21:42:30Z)
• Empowering Private Tutoring by Chaining Large Language Models [87.76985829144834]
  This work explores the development of a full-fledged intelligent tutoring system powered by state-of-the-art large language models (LLMs) The system is into three inter-connected core processes-interaction, reflection, and reaction. Each process is implemented by chaining LLM-powered tools along with dynamically updated memory modules.
  arXiv  Detail & Related papers  (2023-09-15T02:42:03Z)
• Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning [43.83422798569986]
  Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and reliable form of assessment. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers. We propose a simple, in-context learning-based solution for automated distractor and corresponding feedback message generation.
  arXiv  Detail & Related papers  (2023-08-07T01:03:04Z)
• MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems [74.73881579517055]
  We propose a framework to generate such dialogues by pairing human teachers with a Large Language Model prompted to represent common student errors. We describe how we use this framework to collect MathDial, a dataset of 3k one-to-one teacher-student tutoring dialogues.
  arXiv  Detail & Related papers  (2023-05-23T21:44:56Z)
• SmartBook: AI-Assisted Situation Report Generation for Intelligence Analysts [55.73424958012229]
  This work identifies intelligence analysts' practices and preferences for AI assistance in situation report generation. We introduce SmartBook, an automated framework designed to generate situation reports from large volumes of news data. Our comprehensive evaluation of SmartBook, encompassing a user study alongside a content review with an editing study, reveals SmartBook's effectiveness in generating accurate and relevant situation reports.
  arXiv  Detail & Related papers  (2023-03-25T03:03:00Z)
• An Analysis of Programming Course Evaluations Before and After the Introduction of an Autograder [1.329950749508442]
  This paper studies the answers to the standardized university evaluation questionnaires of foundational computer science courses which recently introduced autograding. We hypothesize how the autograder might have contributed to the significant changes in the data, such as, improved interactions between tutors and students, improved overall course quality, improved learning success, increased time spent, and reduced difficulty. The autograder technology can be validated as a teaching method to improve student satisfaction with programming courses.
  arXiv  Detail & Related papers  (2021-10-28T14:09:44Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems [4.716555240531893]
  We explore creating automated, personalized feedback in an intelligent tutoring system (ITS) Our goal is to pinpoint correct and incorrect concepts in student answers in order to achieve better student learning gains.
  arXiv  Detail & Related papers  (2021-03-13T20:33:10Z)
• Educational Question Mining At Scale: Prediction, Analysis and Personalization [35.42197158180065]
  We propose a framework for mining insights from educational questions at scale. We utilize the state-of-the-art Bayesian deep learning method, in particular partial variational auto-encoders (p-VAE) We apply our proposed framework to a real-world dataset with tens of thousands of questions and tens of millions of answers from an online education platform.
  arXiv  Detail & Related papers  (2020-03-12T19:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.