Gamifying Testing in IntelliJ: A Replicability Study

• URL: http://arxiv.org/abs/2504.19294v1
• Date: Sun, 27 Apr 2025 16:17:11 GMT
• Title: Gamifying Testing in IntelliJ: A Replicability Study
• Authors: Philipp Straubinger, Tommaso Fulcini, Giacomo Garaccione, Luca Ardito, Gordon Fraser,
• Abstract summary: Gamification is an emerging technique to enhance motivation and performance in traditionally unengaging tasks like software testing.<n>Previous studies have indicated that gamified systems have the potential to improve software testing processes by providing testers with achievements and feedback.<n>This paper aims to replicate and validate the effects of IntelliGame, a gamification plugin for IntelliJ IDEA to engage developers in writing and executing tests.
• Score: 8.689182960457137
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Gamification is an emerging technique to enhance motivation and performance in traditionally unengaging tasks like software testing. Previous studies have indicated that gamified systems have the potential to improve software testing processes by providing testers with achievements and feedback. However, further evidence of these benefits across different environments, programming languages, and participant groups is required. This paper aims to replicate and validate the effects of IntelliGame, a gamification plugin for IntelliJ IDEA to engage developers in writing and executing tests. The objective is to generalize the benefits observed in earlier studies to new contexts, i.e., the TypeScript programming language and a larger participant pool. The replicability study consists of a controlled experiment with 174 participants, divided into two groups: one using IntelliGame and one with no gamification plugin. The study employed a two-group experimental design to compare testing behavior, coverage, mutation scores, and participant feedback between the groups. Data was collected through test metrics and participant surveys, and statistical analysis was performed to determine the statistical significance. Participants using IntelliGame showed higher engagement and productivity in testing practices than the control group, evidenced by the creation of more tests, increased frequency of executions, and enhanced utilization of testing tools. This ultimately led to better code implementations, highlighting the effectiveness of gamification in improving functional outcomes and motivating users in their testing endeavors. The replication study confirms that gamification, through IntelliGame, positively impacts software testing behavior and developer engagement in coding tasks.

Related papers

• On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementations [53.0667196725616]
  Deep Reinforcement Learning (DRL) is a paradigm of artificial intelligence where an agent uses a neural network to learn which actions to take in a given environment.<n>DRL has recently gained traction from being able to solve complex environments like driving simulators, 3D robotic control, and multiplayer-online-battle-arena video games.<n>Numerous implementations of the state-of-the-art algorithms responsible for training these agents, like the Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) algorithms, currently exist.
  arXiv  Detail & Related papers  (2025-03-28T16:25:06Z)
• Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests [89.09172401497213]
  We examine three evaluation paradigms: large question-answering benchmarks, interactive games, and cognitive tests.<n>We compile a suite of targeted tests that measure cognitive abilities deemed essential for effective language use.<n>Our analyses reveal that interactive games are superior to standard benchmarks in discriminating models.
  arXiv  Detail & Related papers  (2025-02-20T08:36:58Z)
• ViUniT: Visual Unit Tests for More Robust Visual Programming [104.55763189099125]
  When models answer correctly, they produce incorrect programs 33% of the time. We propose Visual Unit Testing (ViUniT), a framework to improve the reliability of visual programs by automatically generating unit tests.
  arXiv  Detail & Related papers  (2024-12-12T01:36:18Z)
• Which Combination of Test Metrics Can Predict Success of a Software Project? A Case Study in a Year-Long Project Course [1.553083901660282]
  Testing plays an important role in securing the success of a software development project. We investigate whether we can quantify the effects various types of testing have on functional suitability.
  arXiv  Detail & Related papers  (2024-08-22T04:23:51Z)
• Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests [4.574205608859157]
  We introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases.
  arXiv  Detail & Related papers  (2024-08-21T15:35:34Z)
• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Gamified GUI testing with Selenium in the IntelliJ IDE: A Prototype Plugin [0.559239450391449]
  This paper presents GIPGUT: a prototype of a gamification plugin for IntelliJ IDEA. The plugin enhances testers' engagement with typically monotonous and tedious tasks through achievements, rewards, and profile customization. The results indicate high usability and positive reception of the gamification elements.
  arXiv  Detail & Related papers  (2024-03-14T20:11:11Z)
• InterEvo-TR: Interactive Evolutionary Test Generation With Readability Assessment [1.6874375111244329]
  We propose incorporating interactive readability assessments made by a tester into EvoSuite. Our approach, InterEvo-TR, interacts with the tester at different moments during the search. Our results show that the strategy to select and present intermediate results is effective for the purpose of readability assessment.
  arXiv  Detail & Related papers  (2024-01-13T13:14:29Z)
• Improving Testing Behavior by Gamifying IntelliJ [13.086283144520513]
  We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment. IntelliGame rewards developers for positive testing behavior using a multi-level achievement system. A controlled experiment with 49 participants reveals substantial differences in the testing behavior triggered by IntelliGame.
  arXiv  Detail & Related papers  (2023-10-17T11:40:55Z)
• Benchopt: Reproducible, efficient and collaborative optimization benchmarks [67.29240500171532]
  Benchopt is a framework to automate, reproduce and publish optimization benchmarks in machine learning. Benchopt simplifies benchmarking for the community by providing an off-the-shelf tool for running, sharing and extending experiments.
  arXiv  Detail & Related papers  (2022-06-27T16:19:24Z)
• StudyMe: A New Mobile App for User-Centric N-of-1 Trials [68.8204255655161]
  N-of-1 trials are multi-crossover self-experiments that allow individuals to systematically evaluate the effect of interventions on their personal health goals. We present StudyMe, an open-source mobile application that is freely available from https://play.google.com/store/apps/details?id=health.studyu.me.
  arXiv  Detail & Related papers  (2021-07-31T20:43:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.