Insights from the Field: Exploring Students' Perspectives on Bad Unit Testing Practices

• URL: http://arxiv.org/abs/2404.10185v1
• Date: Mon, 15 Apr 2024 23:54:45 GMT
• Title: Insights from the Field: Exploring Students' Perspectives on Bad Unit Testing Practices
• Authors: Anthony Peruma, Eman Abdullah AlOmar, Wajdi Aljedaani, Christian D. Newman, Mohamed Wiem Mkaouer,
• Abstract summary: Students might inadvertently deviate from established unit testing best practices, and introduce problematic code into their test suites. Students report on the plugin's usefulness in learning about and detecting test smells, they also identify specific test smells that they consider harmless. We anticipate that our findings will support academia in refining course curricula on unit testing and enabling educators to support students with code review strategies of test code.
• Score: 16.674156958233855
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Educating students about software testing practices is integral to the curricula of many computer science-related courses and typically involves students writing unit tests. Similar to production/source code, students might inadvertently deviate from established unit testing best practices, and introduce problematic code, referred to as test smells, into their test suites. Given the extensive catalog of test smells, it becomes challenging for students to identify test smells in their code, especially for those who lack experience with testing practices. In this experience report, we aim to increase students' awareness of bad unit testing practices, and detail the outcomes of having 184 students from three higher educational institutes utilize an IDE plugin to automatically detect test smells in their code. Our findings show that while students report on the plugin's usefulness in learning about and detecting test smells, they also identify specific test smells that they consider harmless. We anticipate that our findings will support academia in refining course curricula on unit testing and enabling educators to support students with code review strategies of test code.

Related papers

• A Block-Based Testing Framework for Scratch [9.390562437823078]
  We introduce a new category of blocks in Scratch that enables the creation of automated tests. With these blocks, students and teachers alike can create tests and receive feedback directly within the Scratch environment.
  arXiv  Detail & Related papers  (2024-10-11T14:11:26Z)
• Do Test and Environmental Complexity Increase Flakiness? An Empirical Study of SAP HANA [47.29324864511411]
  Flaky tests fail seemingly at random without changes to the code. We study characteristics of tests and the test environment that potentially impact test flakiness.
  arXiv  Detail & Related papers  (2024-09-16T07:52:09Z)
• Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests [4.574205608859157]
  We introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases.
  arXiv  Detail & Related papers  (2024-08-21T15:35:34Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency [45.6224547703717]
  This study focuses on tests of silent sentence reading efficiency, used to assess students' reading ability over time. We propose to fine-tune large language models (LLMs) to simulate how previous students would have responded to unseen items. We show the generated tests closely correspond to the original test's difficulty and reliability based on crowdworker responses.
  arXiv  Detail & Related papers  (2023-10-10T17:59:51Z)
• Unit Testing Challenges with Automated Marking [4.56877715768796]
  We introduce online unit testing challenges with automated marking as a learning tool via the EdStem platform. Results from 92 participants showed that our unit testing challenges have kept students more engaged and motivated. These results inform educators that the online unit testing challenges with automated marking improve overall student learning experience.
  arXiv  Detail & Related papers  (2023-10-10T04:52:44Z)
• Prompting Code Interpreter to Write Better Unit Tests on Quixbugs Functions [0.05657375260432172]
  Unit testing is a commonly-used approach in software engineering to test the correctness and robustness of written code. In this study, we explore the effect of different prompts on the quality of unit tests generated by Code Interpreter. We find that the quality of the generated unit tests is not sensitive to changes in minor details in the prompts provided.
  arXiv  Detail & Related papers  (2023-09-30T20:36:23Z)
• Active Teacher for Semi-Supervised Object Detection [80.10937030195228]
  We propose a novel algorithm called Active Teacher for semi-supervised object detection (SSOD) Active Teacher extends the teacher-student framework to an iterative version, where the label set is partially and gradually augmented by evaluating three key factors of unlabeled examples. With this design, Active Teacher can maximize the effect of limited label information while improving the quality of pseudo-labels.
  arXiv  Detail & Related papers  (2023-03-15T03:59:27Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• How do students test software units? [3.6748639131154315]
  We asked students to fill in a small survey, to do four exercises and to fill a second survey. We interviewed eleven students in semi-structured interviews, to obtain more in-depth insight. One of the misconceptions we found is that most students can only think of test cases based on programming code. Even if no code was provided (black-box testing), students try to come up with code to base their test cases on.
  arXiv  Detail & Related papers  (2021-02-16T07:02:59Z)
• Beyond Accuracy: Behavioral Testing of NLP models with CheckList [66.42971817954806]
  CheckList is a task-agnostic methodology for testing NLP models. CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation. In a user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.
  arXiv  Detail & Related papers  (2020-05-08T15:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.