Improving Testing Behavior by Gamifying IntelliJ
- URL: http://arxiv.org/abs/2310.11171v1
- Date: Tue, 17 Oct 2023 11:40:55 GMT
- Title: Improving Testing Behavior by Gamifying IntelliJ
- Authors: Philipp Straubinger, Gordon Fraser
- Abstract summary: We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment.
IntelliGame rewards developers for positive testing behavior using a multi-level achievement system.
A controlled experiment with 49 participants reveals substantial differences in the testing behavior triggered by IntelliGame.
- Score: 13.086283144520513
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Testing is an important aspect of software development, but unfortunately, it
is often neglected. While test quality analyses such as code coverage or
mutation analysis inform developers about the quality of their tests, such
reports are viewed only sporadically during continuous integration or code
review, if they are considered at all, and their impact on the developers'
testing behavior therefore tends to be negligible. To actually influence
developer behavior, it may rather be necessary to motivate developers directly
within their programming environment, while they are coding. We introduce
IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated
Development Environment, which rewards developers for positive testing behavior
using a multi-level achievement system: A total of 27 different achievements,
each with incremental levels, provide affirming feedback when developers
exhibit commendable testing behavior, and provide an incentive to further
continue and improve this behavior. A controlled experiment with 49
participants given a Java programming task reveals substantial differences in
the testing behavior triggered by IntelliGame: Incentivized developers write
more tests, achieve higher coverage and mutation scores, run their tests more
often, and achieve functionality earlier.
Related papers
- Leveraging Large Language Models for Efficient Failure Analysis in Game Development [47.618236610219554]
This paper proposes a new approach to automatically identify which change in the code caused a test to fail.
The method leverages Large Language Models (LLMs) to associate error messages with the corresponding code changes causing the failure.
Our approach reaches an accuracy of 71% in our newly created dataset, which comprises issues reported by developers at EA over a period of one year.
arXiv Detail & Related papers (2024-06-11T09:21:50Z) - Elevating Software Quality in Agile Environments: The Role of Testing Professionals in Unit Testing [0.0]
Testing is an essential quality activity in the software development process.
This paper explores the participation of test engineers in unit testing within an industrial context.
arXiv Detail & Related papers (2024-03-20T00:41:49Z) - Gamified GUI testing with Selenium in the IntelliJ IDE: A Prototype Plugin [0.559239450391449]
This paper presents GIPGUT: a prototype of a gamification plugin for IntelliJ IDEA.
The plugin enhances testers' engagement with typically monotonous and tedious tasks through achievements, rewards, and profile customization.
The results indicate high usability and positive reception of the gamification elements.
arXiv Detail & Related papers (2024-03-14T20:11:11Z) - DevBench: A Comprehensive Benchmark for Software Development [72.24266814625685]
DevBench is a benchmark that evaluates large language models (LLMs) across various stages of the software development lifecycle.
Empirical studies show that current LLMs, including GPT-4-Turbo, fail to solve the challenges presented within DevBench.
Our findings offer actionable insights for the future development of LLMs toward real-world programming applications.
arXiv Detail & Related papers (2024-03-13T15:13:44Z) - Understanding and Evaluating Developer Behaviour in Programming Tasks [0.0]
In a series of three studies we investigated the specific behaviour of developers solving a specific programming task.
We focused on which source code files they visited, how they related pieces of code and knowledge to others and when and how successful they performed code edits.
arXiv Detail & Related papers (2024-03-13T12:46:42Z) - Observation-based unit test generation at Meta [52.4716552057909]
TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution.
TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults.
Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
arXiv Detail & Related papers (2024-02-09T00:34:39Z) - Gamifying a Software Testing Course with Continuous Integration [13.086283144520513]
Gamekins is a tool that is seamlessly integrated into the Jenkins continuous integration platform.
Developers can earn points by completing test challenges and quests generated by Gamekins.
We observe a correlation between how students test their code and their use of Gamekins.
arXiv Detail & Related papers (2024-01-31T11:00:16Z) - A Survey on What Developers Think About Testing [13.086283144520513]
We conducted a comprehensive survey with 21 questions aimed at assessing developers' current engagement with testing.
We uncover reasons that positively and negatively impact developers' motivation to test.
One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts.
arXiv Detail & Related papers (2023-09-03T12:18:41Z) - CONCORD: Clone-aware Contrastive Learning for Source Code [64.51161487524436]
Self-supervised pre-training has gained traction for learning generic code representations valuable for many downstream SE tasks.
We argue that it is also essential to factor in how developers code day-to-day for general-purpose representation learning.
In particular, we propose CONCORD, a self-supervised, contrastive learning strategy to place benign clones closer in the representation space while moving deviants further apart.
arXiv Detail & Related papers (2023-06-05T20:39:08Z) - Teaching Large Language Models to Self-Debug [62.424077000154945]
Large language models (LLMs) have achieved impressive performance on code generation.
We propose Self- Debugging, which teaches a large language model to debug its predicted program via few-shot demonstrations.
arXiv Detail & Related papers (2023-04-11T10:43:43Z) - SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video
Games Using Risk Based Testing and Machine Learning [62.997667081978825]
Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems.
We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub.
The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
arXiv Detail & Related papers (2022-03-10T00:47:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.