Improving Testing Behavior by Gamifying IntelliJ

• URL: http://arxiv.org/abs/2310.11171v1
• Date: Tue, 17 Oct 2023 11:40:55 GMT
• Title: Improving Testing Behavior by Gamifying IntelliJ
• Authors: Philipp Straubinger, Gordon Fraser
• Abstract summary: We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment. IntelliGame rewards developers for positive testing behavior using a multi-level achievement system. A controlled experiment with 49 participants reveals substantial differences in the testing behavior triggered by IntelliGame.
• Score: 13.086283144520513
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Testing is an important aspect of software development, but unfortunately, it is often neglected. While test quality analyses such as code coverage or mutation analysis inform developers about the quality of their tests, such reports are viewed only sporadically during continuous integration or code review, if they are considered at all, and their impact on the developers' testing behavior therefore tends to be negligible. To actually influence developer behavior, it may rather be necessary to motivate developers directly within their programming environment, while they are coding. We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment, which rewards developers for positive testing behavior using a multi-level achievement system: A total of 27 different achievements, each with incremental levels, provide affirming feedback when developers exhibit commendable testing behavior, and provide an incentive to further continue and improve this behavior. A controlled experiment with 49 participants given a Java programming task reveals substantial differences in the testing behavior triggered by IntelliGame: Incentivized developers write more tests, achieve higher coverage and mutation scores, run their tests more often, and achieve functionality earlier.

Related papers

• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• Engaging Developers in Exploratory Unit Testing through Gamification [11.077232808482128]
  We show challenges and quests generated by the Gamekins system to make testing more engaging and seamlessly blend it with regular coding tasks. In a 60-minute experiment, we evaluated Gamekins' impact on test suite quality and bug detection. Results show that participants actively interacted with the tool, achieving nearly 90% line coverage and detecting 11 out of 14 bugs.
  arXiv  Detail & Related papers  (2024-08-09T08:00:41Z)
• Leveraging Large Language Models for Efficient Failure Analysis in Game Development [47.618236610219554]
  This paper proposes a new approach to automatically identify which change in the code caused a test to fail. The method leverages Large Language Models (LLMs) to associate error messages with the corresponding code changes causing the failure. Our approach reaches an accuracy of 71% in our newly created dataset, which comprises issues reported by developers at EA over a period of one year.
  arXiv  Detail & Related papers  (2024-06-11T09:21:50Z)
• Gamified GUI testing with Selenium in the IntelliJ IDE: A Prototype Plugin [0.559239450391449]
  This paper presents GIPGUT: a prototype of a gamification plugin for IntelliJ IDEA. The plugin enhances testers' engagement with typically monotonous and tedious tasks through achievements, rewards, and profile customization. The results indicate high usability and positive reception of the gamification elements.
  arXiv  Detail & Related papers  (2024-03-14T20:11:11Z)
• Understanding and Evaluating Developer Behaviour in Programming Tasks [0.0]
  In a series of three studies we investigated the specific behaviour of developers solving a specific programming task. We focused on which source code files they visited, how they related pieces of code and knowledge to others and when and how successful they performed code edits.
  arXiv  Detail & Related papers  (2024-03-13T12:46:42Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Gamifying a Software Testing Course with Continuous Integration [13.086283144520513]
  Gamekins is a tool that is seamlessly integrated into the Jenkins continuous integration platform. Developers can earn points by completing test challenges and quests generated by Gamekins. We observe a correlation between how students test their code and their use of Gamekins.
  arXiv  Detail & Related papers  (2024-01-31T11:00:16Z)
• A Survey on What Developers Think About Testing [13.086283144520513]
  We conducted a comprehensive survey with 21 questions aimed at assessing developers' current engagement with testing. We uncover reasons that positively and negatively impact developers' motivation to test. One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts.
  arXiv  Detail & Related papers  (2023-09-03T12:18:41Z)
• CONCORD: Clone-aware Contrastive Learning for Source Code [64.51161487524436]
  Self-supervised pre-training has gained traction for learning generic code representations valuable for many downstream SE tasks. We argue that it is also essential to factor in how developers code day-to-day for general-purpose representation learning. In particular, we propose CONCORD, a self-supervised, contrastive learning strategy to place benign clones closer in the representation space while moving deviants further apart.
  arXiv  Detail & Related papers  (2023-06-05T20:39:08Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video Games Using Risk Based Testing and Machine Learning [62.997667081978825]
  Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems. We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub. The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
  arXiv  Detail & Related papers  (2022-03-10T00:47:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.