RippleGUItester: Change-Aware Exploratory Testing

• URL: http://arxiv.org/abs/2603.03121v1
• Date: Tue, 03 Mar 2026 15:56:49 GMT
• Title: RippleGUItester: Change-Aware Exploratory Testing
• Abstract summary: We present RippleGUItester, a change-driven testing system that treats a code change as the epicenter of a ripple effect.<n>We evaluate our approach on hundreds of real-world code changes across four widely used software systems: Firefox, Zettlr, JabRef, and Godot.<n>In total, we identify 26 previously unknown bugs that still exist in the latest versions of the evaluated systems.
• Score: 22.40767870807728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Software systems evolve continuously through frequent code changes, yet such changes often introduce unintended bugs despite extensive testing and code review. Existing testing approaches are largely constrained to predefined execution paths or rely on unguided exploration, leaving many change-induced issues undetected. To address this challenge, we present RippleGUItester, a change-driven testing system that treats a code change as the epicenter of a ripple effect and explores its broader, user-visible impacts via the GUI. Given a code change, RippleGUItester performs LLM-based change-impact analysis to generate and enrich realistic test scenarios, executes these scenarios on both pre-change and post-change versions of the system, and applies differential analysis to identify behavioral differences. Crucially, RippleGUItester employs multimodal bug detection, comparing visual GUI changes and interpreting them in the context of natural-language change intents to distinguish unintended bugs from intended behavioral updates. We evaluate our approach on hundreds of real-world code changes across four widely used software systems: Firefox, Zettlr, JabRef, and Godot. Our results show that the proposed approach uncovers bugs introduced by code changes that were missed by existing test suites, CI pipelines, and code review. In total, we identify 26 previously unknown bugs that still exist in the latest versions of the evaluated systems. After reporting, 16 bugs have been fixed, 2 have been confirmed, 6 are still under discussion, and 2 were marked as intended. We envision RippleGUItester being applied before or shortly after a code change is merged, enabling earlier detection of regressions.

Related papers

• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• DiffTestGen: Change-Directed LLM-Based Testing for Exposing Behavioral Differences [12.692695524978332]
  This paper presentsDiffTestGen, a change-directed, LLM-based differential testing approach.<n>We show that DiffTestGen exposes behavioral differences in 78.2% of the PRs while achieving an average union coverage of 90.7%.<n>By integrating DiffTestGen with the Testora regression detector, we show that the identified behavioral differences can be used to detect regression bugs missed by the best existing approaches.
  arXiv  Detail & Related papers  (2026-07-17T14:57:59Z)
• Enhancing Code Review through Fuzzing and Likely Invariants [13.727241655311664]
  We present FuzzSight, a framework that leverages likely invariants from non-crashing fuzzing inputs to highlight behavioral differences across program versions.<n>In our evaluation, FuzzSight flagged 75% of regression bugs and up to 80% of vulnerabilities uncovered by 24-hour fuzzing.
  arXiv  Detail & Related papers  (2025-10-17T10:30:22Z)
• Probing Pre-trained Language Models on Code Changes: Insights from ReDef, a High-Confidence Just-in-Time Defect Prediction Dataset [0.0]
  We present ReDef, a high-confidence benchmark of function-level modifications curated from 22 large-scale C/C++ projects.<n>Defective cases are anchored by revert commits, while clean cases are validated through post-hoc history checks.<n>This pipeline yields 3,164 defective and 10,268 clean modifications, offering substantially more reliable labels than prior existing resources.
  arXiv  Detail & Related papers  (2025-09-11T07:07:11Z)
• Testora: Using Natural Language Intent to Detect Behavioral Regressions [16.852583199679117]
  This paper presents Testora, the first automated approach that detects regressions by comparing the intentions of a code change against behavioral differences caused by the code change.<n>We find 19 regression bugs and 11 PRs that, despite having another intention, coincidentally fix a bug.<n>We envision our approach to be used before or shortly after a code change gets merged into a code base, providing a way to early on detect regressions that are not caught by traditional approaches.
  arXiv  Detail & Related papers  (2025-03-24T11:55:35Z)
• Detect Changes like Humans: Incorporating Semantic Priors for Improved Change Detection [52.62459671461816]
  This paper explores incorporating semantic priors from visual foundation models to improve the ability to detect changes.<n>Inspired by the human visual paradigm, a novel dual-stream feature decoder is derived to distinguish changes by combining semantic-aware features and difference-aware features.
  arXiv  Detail & Related papers  (2024-12-22T08:27:15Z)
• Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detection [82.65760006883248]
  We introduce a new task named Change Detection Question Answering and Grounding (CDQAG) CDQAG extends the traditional change detection task by providing interpretable textual answers and intuitive visual evidence. We construct the first CDQAG benchmark dataset, termed QAG-360K, comprising over 360K triplets of questions, textual answers, and corresponding high-quality visual masks.
  arXiv  Detail & Related papers  (2024-10-31T11:20:13Z)
• Seeing is Believing: Vision-driven Non-crash Functional Bug Detection for Mobile Apps [26.96558418166514]
  This paper proposes a novel vision-driven, multi-agent collaborative automated GUI testing approach for detecting non-crash functional bugs.<n>We evaluate Trident on 590 non-crash bugs and compare it with 12 baselines, it can achieve more than 14%-112% and 108%-147% boost in average recall and precision.
  arXiv  Detail & Related papers  (2024-07-03T11:58:09Z)
• Leveraging Large Language Models for Efficient Failure Analysis in Game Development [47.618236610219554]
  This paper proposes a new approach to automatically identify which change in the code caused a test to fail. The method leverages Large Language Models (LLMs) to associate error messages with the corresponding code changes causing the failure. Our approach reaches an accuracy of 71% in our newly created dataset, which comprises issues reported by developers at EA over a period of one year.
  arXiv  Detail & Related papers  (2024-06-11T09:21:50Z)
• MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations [50.79913333804232]
  We propose a memory-supported transformer (MS-Former) for weakly supervised change detection. MS-Former consists of a bi-directional attention block (BAB) and a patch-level supervision scheme (PSS) Experimental results on three benchmark datasets demonstrate the effectiveness of our proposed method in the change detection task.
  arXiv  Detail & Related papers  (2023-11-16T09:57:29Z)
• Identifying Defect-Inducing Changes in Visual Code [54.20154707138088]
  "SZZ Visual Code" (SZZ-VC) is an algorithm that finds changes in visual code based on the differences of graphical elements rather than differences of lines to detect defect-inducing changes. We validated the algorithm for an industry-made AAA video game and 20 music visual programming defects across 12 open source projects.
  arXiv  Detail & Related papers  (2023-09-07T00:12:28Z)
• Learning Sinkhorn divergences for supervised change point detection [24.30834981766022]
  We present a novel change point detection framework that uses true change point instances as supervision for learning a ground metric. Our method can be used to learn a sparse metric which can be useful for both feature selection and interpretation.
  arXiv  Detail & Related papers  (2022-02-08T17:11:40Z)
• D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis [55.15995704119158]
  We propose D2A, a differential analysis based approach to label issues reported by static analysis tools. We use D2A to generate a large labeled dataset to train models for vulnerability identification.
  arXiv  Detail & Related papers  (2021-02-16T07:46:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.