Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions

• URL: http://arxiv.org/abs/2505.10913v1
• Date: Fri, 16 May 2025 06:32:51 GMT
• Title: Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions
• Authors: Muntasir Hoq, Ananya Rao, Reisha Jaishankar, Krish Piryani, Nithya Janapati, Jessica Vandenberg, Bradford Mott, Narges Norouzi, James Lester, Bita Akram,
• Abstract summary: This paper presents a scalable framework for automatically detecting logical errors in students' programming solutions.<n>Our framework is based on an explainable Abstract Syntax Tree (AST) embedding model, the Subtree-based Attention Neural Network (SANN)
• Score: 4.0782995609938
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In Computer Science (CS) education, understanding factors contributing to students' programming difficulties is crucial for effective learning support. By identifying specific issues students face, educators can provide targeted assistance to help them overcome obstacles and improve learning outcomes. While identifying sources of struggle, such as misconceptions, in real-time can be challenging in current educational practices, analyzing logical errors in students' code can offer valuable insights. This paper presents a scalable framework for automatically detecting logical errors in students' programming solutions. Our framework is based on an explainable Abstract Syntax Tree (AST) embedding model, the Subtree-based Attention Neural Network (SANN), that identifies the structural components of programs containing logical errors. We conducted a series of experiments to evaluate its effectiveness, and the results suggest that our framework can accurately capture students' logical errors and, more importantly, provide us with deeper insights into their learning processes, offering a valuable tool for enhancing programming education.

Related papers

• CoderAgent: Simulating Student Behavior for Personalized Programming Learning with Large Language Models [34.62411261398559]
  We propose a LLM-based agent, CoderAgent, to simulate students' programming processes in a fine-grained manner without relying on real data.<n>Specifically, we equip each human learner with an intelligent agent, the core of which lies in capturing the cognitive states of the human programming practice process.
  arXiv  Detail & Related papers  (2025-05-27T02:43:38Z)
• From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning [76.09281171131941]
  Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy.<n>We propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors.
  arXiv  Detail & Related papers  (2025-05-21T15:00:07Z)
• Example-Based Learning in Software Engineering Education: A Systematic Mapping Study [0.43012765978447565]
  Example-Based Learning (EBL) has shown promise in improving the quality of Software Engineering Education (SEE)<n>This study aims to investigate and classify the existing empirical evidence about using EBL in SEE.
  arXiv  Detail & Related papers  (2025-03-23T14:14:25Z)
• ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models [81.12673534903979]
  Tool learning has emerged as a crucial capability for large language models (LLMs) to solve complex real-world tasks through interaction with external tools.<n>We propose ToolCoder, a novel framework that reformulates tool learning as a code generation task.
  arXiv  Detail & Related papers  (2025-02-17T03:42:28Z)
• Knowledge Tracing in Programming Education Integrating Students' Questions [0.0]
  This paper introduces SQKT (Students' Question-based Knowledge Tracing), a knowledge tracing model that leverages students' questions and automatically extracted skill information.<n> Experimental results demonstrate SQKT's superior performance in predicting student completion across various Python programming courses of differing difficulty levels.<n> SQKT can be used to tailor educational content to individual learning needs and design adaptive learning systems in computer science education.
  arXiv  Detail & Related papers  (2025-01-22T14:13:40Z)
• LLM-based Cognitive Models of Students with Misconceptions [55.29525439159345]
  This paper investigates whether Large Language Models (LLMs) can be instruction-tuned to meet this dual requirement. We introduce MalAlgoPy, a novel Python library that generates datasets reflecting authentic student solution patterns. Our insights enhance our understanding of AI-based student models and pave the way for effective adaptive learning systems.
  arXiv  Detail & Related papers  (2024-10-16T06:51:09Z)
• AI-Driven Virtual Teacher for Enhanced Educational Efficiency: Leveraging Large Pretrain Models for Autonomous Error Analysis and Correction [21.159378560503036]
  This paper introduces an innovative textbfVirtual textbfAI textbfTeacher system designed to autonomously analyze and correct student textbfErrors (VATE)<n>The system has been deployed on the Squirrel AI learning platform for elementary mathematics education, where it achieves 78.3% accuracy in error analysis.
  arXiv  Detail & Related papers  (2024-09-14T10:27:36Z)
• When Do Program-of-Thoughts Work for Reasoning? [51.2699797837818]
  We propose complexity-impacted reasoning score (CIRS) to measure correlation between code and reasoning abilities. Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity. Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.
  arXiv  Detail & Related papers  (2023-08-29T17:22:39Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• LogiGAN: Learning Logical Reasoning via Adversarial Pre-training [58.11043285534766]
  We present LogiGAN, an unsupervised adversarial pre-training framework for improving logical reasoning abilities of language models. Inspired by the facilitation effect of reflective thinking in human learning, we simulate the learning-thinking process with an adversarial Generator-Verifier architecture. Both base and large size language models pre-trained with LogiGAN demonstrate obvious performance improvement on 12 datasets.
  arXiv  Detail & Related papers  (2022-05-18T08:46:49Z)
• Toward Semi-Automatic Misconception Discovery Using Code Embeddings [4.369757255496184]
  We present a novel method for the semi-automated discovery of problem-specific misconceptions from students' program code in computing courses. We trained the model on a block-based programming dataset and used the learned embedding to cluster incorrect student submissions.
  arXiv  Detail & Related papers  (2021-03-07T20:32:41Z)
• Automating Program Structure Classification [6.215059642140581]
  We show how supervised machine learning methods can automatically classify student programs into a predetermined set of high-level structures. We demonstrate that these models can achieve 91% classification accuracy when trained on 108 programs.
  arXiv  Detail & Related papers  (2021-01-15T21:24:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.