Designing for Novice Debuggers: A Pilot Study on an AI-Assisted Debugging Tool

• URL: http://arxiv.org/abs/2509.21067v2
• Date: Thu, 02 Oct 2025 04:41:48 GMT
• Title: Designing for Novice Debuggers: A Pilot Study on an AI-Assisted Debugging Tool
• Authors: Oka Kurniawan, Erick Chandra, Christopher M. Poskitt, Yannic Noller, Kenny Tsu Wei Choo, Cyrille Jegourel,
• Abstract summary: We present findings from our second design debugger, which we tested with a group of undergraduate students.<n>Our results indicate that the students found the tool highly effective in resolving semantic errors and significantly easier to use than the first version.<n>We conclude that any AI-assisted debug approach should be personalized based on user profiles to optimize their interactions with the tool.
• Score: 5.192564039251338
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Debugging is a fundamental skill that novice programmers must develop. Numerous tools have been created to assist novice programmers in this process. Recently, large language models (LLMs) have been integrated with automated program repair techniques to generate fixes for students' buggy code. However, many of these tools foster an over-reliance on AI and do not actively engage students in the debugging process. In this work, we aim to design an intuitive debugging assistant, CodeHinter, that combines traditional debugging tools with LLM-based techniques to help novice debuggers fix semantic errors while promoting active engagement in the debugging process. We present findings from our second design iteration, which we tested with a group of undergraduate students. Our results indicate that the students found the tool highly effective in resolving semantic errors and significantly easier to use than the first version. Consistent with our previous study, error localization was the most valuable feature. Finally, we conclude that any AI-assisted debugging approach should be personalized based on user profiles to optimize their interactions with the tool.

Related papers

• Enhancing Debugging Skills with AI-Powered Assistance: A Real-Time Tool for Debugging Support [8.607022377771422]
  It offers real-time support by analyzing code, suggesting breakpoints, and providing contextual hints.<n>Using RAG with LLMs, program slicing, and customs, it enhances efficiency by minimizing LLM calls and improving accuracy.
  arXiv  Detail & Related papers  (2026-01-05T19:20:59Z)
• Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning [63.071280297939005]
  We present Transductive Visual Programming (TVP), a novel framework that builds new tools from its own experience rather than speculation.<n>TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%.<n>Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual programming agents.
  arXiv  Detail & Related papers  (2025-12-24T04:30:21Z)
• Do AI models help produce verified bug fixes? [62.985237003585674]
  Large Language Models are used to produce corrections to software bugs.<n>This paper investigates how programmers use Large Language Models to complement their own skills.<n>The results are a first step towards a proper role for AI and LLMs in providing guaranteed-correct fixes to program bugs.
  arXiv  Detail & Related papers  (2025-07-21T17:30:16Z)
• ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models [81.12673534903979]
  Tool learning has emerged as a crucial capability for large language models (LLMs) to solve complex real-world tasks through interaction with external tools.<n>We propose ToolCoder, a novel framework that reformulates tool learning as a code generation task.
  arXiv  Detail & Related papers  (2025-02-17T03:42:28Z)
• Simulated Interactive Debugging [7.3742419367796535]
  We present our approach called Simulated Interactive that interactively guides students along the debug process.<n>The guidance aims to empower the students to repair their solutions and have a proper learning experience.<n>We developed an implementation using traditional fault localization techniques and large language models.
  arXiv  Detail & Related papers  (2025-01-16T17:47:18Z)
• Learning to Ask: When LLM Agents Meet Unclear Instruction [55.65312637965779]
  Large language models (LLMs) can leverage external tools for addressing a range of tasks unattainable through language skills alone.<n>We evaluate the performance of LLMs tool-use under imperfect instructions, analyze the error patterns, and build a challenging tool-use benchmark called Noisy ToolBench.<n>We propose a novel framework, Ask-when-Needed (AwN), which prompts LLMs to ask questions to users whenever they encounter obstacles due to unclear instructions.
  arXiv  Detail & Related papers  (2024-08-31T23:06:12Z)
• A Proposal for a Debugging Learning Support Environment for Undergraduate Students Majoring in Computer Science [0.0]
  Students do not know how to use a debugger or have never used one. We implemented a function in Scratch that allows for self-learning of correct breakpoint placement.
  arXiv  Detail & Related papers  (2024-07-25T03:34:19Z)
• Code Compass: A Study on the Challenges of Navigating Unfamiliar Codebases [2.808331566391181]
  We propose a novel tool, Code, to address these issues. Our study highlights a significant gap in current tools and methodologies. Our formative study demonstrates how effectively the tool reduces the time developers spend navigating documentation.
  arXiv  Detail & Related papers  (2024-05-10T06:58:31Z)
• TOOLVERIFIER: Generalization to New Tools via Self-Verification [69.85190990517184]
  We introduce a self-verification method which distinguishes between close candidates by self-asking contrastive questions during tool selection. Experiments on 4 tasks from the ToolBench benchmark, consisting of 17 unseen tools, demonstrate an average improvement of 22% over few-shot baselines.
  arXiv  Detail & Related papers  (2024-02-21T22:41:38Z)
• A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges [23.467373994306524]
  In practice, developers do not accept AI programming assistants' initial suggestions at a high frequency. To understand developers' practices while using these tools, we administered a survey to a large population of developers. We found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax. We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements.
  arXiv  Detail & Related papers  (2023-03-30T03:21:53Z)
• ART: Automatic multi-step reasoning and tool-use for large language models [105.57550426609396]
  Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings. Each reasoning step can rely on external tools to support computation beyond the core LLM capabilities. We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program.
  arXiv  Detail & Related papers  (2023-03-16T01:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.