Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing

• URL: http://arxiv.org/abs/2607.00929v1
• Date: Wed, 01 Jul 2026 13:32:42 GMT
• Title: Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing
• Abstract summary: This paper proposes a novel approach, DDMT, to enhance the applicability of delta debug.<n>Our key insight is to redesign an oracle-independent test function and incorporate it into the reduction procedure.<n>The experiments evaluate DDMT on 66 subjects across both oracle-available and oracle-deficient scenarios.
• Score: 4.58866559719705
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Delta debugging provides an automatic way to minimize a program input while preserving a certain property. However, its effectiveness fundamentally relies on the availability of test oracles to determine whether a reduced input still preserves the specific property. Consequently, the oracle problem substantially limits the applicability of existing delta debugging techniques, particularly for oracle-deficient programs where output correctness cannot be directly determined. To address this problem, this paper proposes a novel approach, DDMT, to enhance the applicability of delta debugging, especially facilitating its application to oracle-deficient programs. Our key insight is to redesign an oracle-independent test function and incorporate it into the reduction procedure of delta debugging such that the property-preservation validation can be accomplished without requiring a test oracle. To this end, DDMT employs the technique of metamorphic testing, which is a property-based and oracle-independent testing method. It establishes a metamorphic testing-based test function, using it as a replacement for the original test function adopted by delta debugging. The experiments evaluate DDMT on 66 subjects across both oracle-available and oracle-deficient scenarios, with different delta debugging approaches. The results positively confirm that DDMT can enhance the applicability of delta debugging while often preserving or improving reduction effectiveness and query efficiency. Furthermore, compared to the relevant delta debugging approaches, DDMT is also able to achieve performance improvements with proper configurations.

Related papers

• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning [82.89535601592739]
  We propose a two-stage pipeline that generates reliable base tool-use trajectories through self-evolving synthesis with multi-level validation.<n>These augmentations introduce distractor tools, indirect or ambiguous user queries, and noisy, multi-format, or erroneous tool outputs.<n>This design enables automatic reward computation via reference matching for standard cases and lightweight judge-assisted verification for special behaviors such as error detection.
  arXiv  Detail & Related papers  (2026-04-10T18:38:52Z)
• Audio-Visual Continual Test-Time Adaptation without Forgetting [28.411642462831647]
  continual test-time adaptation involves continually adapting a source audio-visual model at test-time.<n>We propose a method, $texttAV-CTTA$, that improves test-time performance of the models without access to any source data.
  arXiv  Detail & Related papers  (2026-02-20T04:55:01Z)
• Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning [63.55145330447408]
  We propose a novel textbfSelf-textbfPerceptinon textbfTuning (textbfSPT) method for anomaly segmentation.<n>The SPT method incorporates a self-drafting tuning strategy, which generates an initial coarse draft of the anomaly mask, followed by a refinement process.
  arXiv  Detail & Related papers  (2024-11-26T08:33:25Z)
• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [69.41389326333771]
  Vision-language foundation models can be unreliable when significant distribution gaps exist between training and test data.<n>We propose DOTA (DistributiOnal Test-time Adaptation), a simple yet effective method addressing this limitation.<n>This distribution-centric approach enables the model to continually learn and adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Active Test-Time Adaptation: Theoretical Analyses and An Algorithm [51.84691955495693]
  Test-time adaptation (TTA) addresses distribution shifts for streaming test data in unsupervised settings. We propose the novel problem setting of active test-time adaptation (ATTA) that integrates active learning within the fully TTA setting.
  arXiv  Detail & Related papers  (2024-04-07T22:31:34Z)
• Validity-Preserving Delta Debugging via Generator Trace Reduction [14.24086822861706]
  GReduce searches for other executions on the generator that yield reduced, valid test inputs.<n>GReduce substantially outperforms state-of-the-art syntax-based reducers including Perses and T-PDD.
  arXiv  Detail & Related papers  (2024-02-07T07:12:27Z)
• Better Practices for Domain Adaptation [62.70267990659201]
  Domain adaptation (DA) aims to provide frameworks for adapting models to deployment data without using labels. Unclear validation protocol for DA has led to bad practices in the literature. We show challenges across all three branches of domain adaptation methodology.
  arXiv  Detail & Related papers  (2023-09-07T17:44:18Z)
• Towards Automatic Generation of Amplified Regression Test Oracles [44.45138073080198]
  We propose a test oracle derivation approach to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour.
  arXiv  Detail & Related papers  (2023-07-28T12:38:44Z)
• Applying and Extending the Delta Debugging Algorithm for Elevator Dispatching Algorithms (Experience Paper) [7.289672463326423]
  In an elevator dispatching algorithm, it is of high benefit to provide the minimal test input to the software developers. In this paper, we enhance this technique by first monitoring the environment at which the CPS operates as well as its physical states. In a second step, we use such identified stable states to help the delta debug algorithm isolate the failure-inducing test inputs more efficiently.
  arXiv  Detail & Related papers  (2023-05-28T19:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.