Enhancing Software Testing Education: Understanding Where Students Struggle

• URL: http://arxiv.org/abs/2510.00957v1
• Date: Wed, 01 Oct 2025 14:30:22 GMT
• Title: Enhancing Software Testing Education: Understanding Where Students Struggle
• Authors: Shiza Andleeb, Teo Mendoza, Lucas Cordova, Gursimran Walia, Jeffrey C. Carver,
• Abstract summary: This study examines the specific testing concepts that lead students to make ineffective changes.<n>We analyzed student submissions from two assignments to identify prevalent conceptual gaps and patterns of unproductive modification.
• Score: 0.5447924312563365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective software testing is critical for producing reliable and secure software, yet many computer science students struggle to master the foundational concepts required to construct comprehensive test suites. While automated feedback tools are widely used to support student learning, it remains unclear which testing concepts are most frequently misunderstood and how these misunderstandings are reflected in students' test suite revisions. This study examines the specific testing concepts that lead students to make ineffective changes, those that fail to improve code coverage, during test suite development. Leveraging an automated feedback tool in a senior-level software testing course, we analyzed student submissions from two assignments to identify prevalent conceptual gaps and patterns of unproductive modification. Our results reveal that decision coverage and exception handling are persistent challenges, and that students most often make superficial or method-level changes that do not enhance coverage. These findings provide actionable insights for educators, researchers, and tool designers. By pinpointing the concepts that most often contribute to poor testing outcomes, we can refine feedback systems, target instruction to address persistent misconceptions, and more effectively support students in developing robust, maintainable test suites.

Related papers

• Existing Large Language Model Unlearning Evaluations Are Inconclusive [105.55899615056573]
  We show that some evaluations introduce substantial new information into the model, potentially masking true unlearning performance.<n>We demonstrate that evaluation outcomes vary significantly across tasks, undermining the generalizability of current evaluation routines.<n>We propose two principles for future unlearning evaluations: minimal information injection and downstream task awareness.
  arXiv  Detail & Related papers  (2025-05-31T19:43:00Z)
• PyEvalAI: AI-assisted evaluation of Jupyter Notebooks for immediate personalized feedback [43.56788158589046]
  PyEvalAI scores Jupyter notebooks using a combination of unit tests and a locally hosted language model to preserve privacy.<n>A case study demonstrates its effectiveness in improving feedback speed and grading efficiency for exercises in a university-level course on numerics.
  arXiv  Detail & Related papers  (2025-02-25T18:20:20Z)
• Chatbots im Schulunterricht: Wir testen das Fobizz-Tool zur automatischen Bewertung von Hausaufgaben [0.0]
  This study examines the AI-powered grading tool "AI Grading Assistant" by the German company Fobizz.<n>The tool's numerical grades and qualitative feedback are often random and do not improve even when its suggestions are incorporated.<n>The study critiques the broader trend of adopting AI as a quick fix for systemic problems in education.
  arXiv  Detail & Related papers  (2024-12-09T16:50:02Z)
• A Feedback Toolkit and Procedural Guidance for Teaching Thorough Testing [0.7829352305480285]
  Correctness is one of the more important criteria of qualitative software.<n>Students do not receive sufficient feedback on code quality and tests unless specified in the assignment.<n>We have developed a procedural guidance that guides students to an implementation with appropriate tests.
  arXiv  Detail & Related papers  (2024-11-30T10:02:57Z)
• A Block-Based Testing Framework for Scratch [9.390562437823078]
  We introduce a new category of blocks in Scratch that enables the creation of automated tests. With these blocks, students and teachers alike can create tests and receive feedback directly within the Scratch environment.
  arXiv  Detail & Related papers  (2024-10-11T14:11:26Z)
• Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
  Large language models (LLMs) present an opportunity to scale high-quality personalized education to all. LLMs struggle to precisely detect student's errors and tailor their feedback to these errors. Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
  arXiv  Detail & Related papers  (2024-07-12T10:11:40Z)
• Unit Testing Challenges with Automated Marking [4.56877715768796]
  We introduce online unit testing challenges with automated marking as a learning tool via the EdStem platform. Results from 92 participants showed that our unit testing challenges have kept students more engaged and motivated. These results inform educators that the online unit testing challenges with automated marking improve overall student learning experience.
  arXiv  Detail & Related papers  (2023-10-10T04:52:44Z)
• Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning [43.83422798569986]
  Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and reliable form of assessment. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers. We propose a simple, in-context learning-based solution for automated distractor and corresponding feedback message generation.
  arXiv  Detail & Related papers  (2023-08-07T01:03:04Z)
• On the Use of Static Analysis to Engage Students with Software Quality Improvement: An Experience with PMD [12.961585735468313]
  We aim to reflect on our experience with teaching the use of static analysis for the purpose of evaluating its effectiveness in helping students with respect to improving software quality. This paper discusses the results of an experiment in the classroom over a period of 3 academic semesters, involving 65 submissions that carried out code review activity of 690 rules using PMD.
  arXiv  Detail & Related papers  (2023-02-11T00:21:04Z)
• Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning [65.12226891589592]
  This paper proposes a new approach to automated game validation and testing. Our method leverages a data-driven imitation learning technique, which requires little effort and time and no knowledge of machine learning or programming.
  arXiv  Detail & Related papers  (2022-08-15T11:08:44Z)
• Continuous Examination by Automatic Quiz Assessment Using Spiral Codes and Image Processing [69.35569554213679]
  Paper quizzes are affordable and within reach of campus education in classrooms. correction of the quiz is a considerable obstacle. We suggest mitigating the issue by a novel image processing technique.
  arXiv  Detail & Related papers  (2022-01-26T22:58:15Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.