Metamorphic Testing of Transpilers via Mutation Consistency of Programs

• URL: http://arxiv.org/abs/2607.29247v1
• Date: Fri, 31 Jul 2026 10:20:08 GMT
• Title: Metamorphic Testing of Transpilers via Mutation Consistency of Programs
• Abstract summary: This paper introduces a novel metamorphic testing technique tailored to transpilers.<n>Instead of reasoning about the runtime behavior of compiled programs, our approach defines metamorphic relations directly over the source code produced by the transpiler.
• Score: 2.064612766965483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transpilers are increasingly used for software development, especially in industrial domains that rely on domain-specific languages (DSLs), to allow engineers to work with familiar concepts and appropriate abstractions. Ensuring the correctness of these instruments is therefore critical in many industrial settings. This paper observes that existing approaches for compiler testing hardly generalize to transpilers. Differential testing approaches are hindered as multiple equivalent implementations of the transpiler under test are seldom available in practice. The approaches based on metamorphic testing assume the ability to execute the compiled binaries, an assumption that cannot be always made for transpilers, which oftentimes produce results expressed as source code, requiring complex toolchains, hardware-in-the-loop setups, and depending on non trivial inputs. This paper introduces a novel metamorphic testing technique tailored to transpilers. Instead of reasoning about the runtime behavior of compiled programs, our approach defines metamorphic relations directly over the source code produced by the transpiler. These relations capture a property that we call mutation consistency of the (transpiled) programs: mutation-style changes in the input DSL program must induce predictable and structurally consistent changes in the generated output. We implemented this idea in a tool, MCP-Tester, and evaluated it through a case study conducted in the context of a technology-transfer project. Our current empirical results indicate that the proposed approach can effectively reveal faults that would remain undetected with pure fuzzing.

Related papers

• Articulating Assumptions in AI-Generated Scientific Analyses through Task Decomposition [0.0]
  We introduce a multi-agent framework for analyzing and comparing scientific programs generated by LLMs.<n>The framework assigns code generation, execution, tracing, and validation to separate agents.<n>It reconstructs how key output quantities are produced and to identify differences between the intended analysis and the implemented code.
  arXiv  Detail & Related papers  (2026-07-07T02:40:32Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• You Don't Need Public Tests to Generate Correct Code [0.9668407688201359]
  We show that large language models possess the capability to autonomously construct valid inputs and simulate execution flows for self-correction.<n>We introduce DryRUN, a framework that removes the necessity for ground-truth data by enabling the LLM to iteratively plan, synthesize its own test inputs, and run simulated executions.
  arXiv  Detail & Related papers  (2026-04-23T12:21:03Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• TENSURE: Fuzzing Sparse Tensor Compilers (Registered Report) [9.083996936429465]
  We present TENSURE, the first black-box fuzzing framework specifically designed for the testing of sparse Compilers.<n>We propose a novel constraint-based generation algorithm that guarantees 100% semantic validity of synthesized kernels.<n>Our evaluation on two state-of-the-art systems, TACO and Finch, reveals widespread fragility, particularly in TACO, where TENSURE exposed crashes or silent miscompilations.
  arXiv  Detail & Related papers  (2026-03-19T00:13:14Z)
• Understanding by Reconstruction: Reversing the Software Development Process for LLM Pretraining [66.89012795621349]
  Large Language Models (LLMs) often struggle with the deep, long-horizon reasoning required for complex software engineering.<n>We propose a novel paradigm: understanding via reconstruction.<n>We introduce a framework that synthesizes latent agentic trajectories using a multi-agent simulation.
  arXiv  Detail & Related papers  (2026-03-11T09:23:20Z)
• Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking [54.43083499412643]
  Test-time algorithms that combine the generative power of language models with process verifiers offer a promising lever for eliciting new reasoning capabilities.<n>We introduce a new process-guided test-time sampling algorithm, VGB, which uses theoretically grounded backtracking to achieve provably better robustness to verifier errors.
  arXiv  Detail & Related papers  (2025-10-03T16:21:14Z)
• A Novel Mutation Based Method for Detecting FPGA Logic Synthesis Tool Bugs [7.8865444084780965]
  We propose VERMEI, a new method for testing FPGA logic synthesis tools.<n> VERMEI consists of three modules: preprocessing, equivalent mutation, and bug identification.<n>Within five months, VERMEI reported 15 bugs to vendors, 9 of which were confirmed as new.
  arXiv  Detail & Related papers  (2025-08-21T13:11:59Z)
• Generating Highly Structured Test Inputs Leveraging Constraint-Guided Graph Refinement [4.121384394709256]
  This study investigates whether test inputs for structured domains can be unified through a graph-based representation.<n>We will evaluate the effectiveness of this approach in enhancing input validity and semantic preservation across eight AI systems.
  arXiv  Detail & Related papers  (2025-07-28T18:54:04Z)
• Diagnostic Spatio-temporal Transformer with Faithful Encoding [54.02712048973161]
  This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex-temporal (ST) dependency. We formalize the problem as supervised dependency discovery, where the ST dependency is learned as a side product of time-series classification. We show that temporal positional encoding used in existing ST transformer works has a serious limitation capturing frequencies in higher frequencies (short time scales) We also propose a new ST dependency discovery framework, which can provide readily consumable diagnostic information in both spatial and temporal directions.
  arXiv  Detail & Related papers  (2023-05-26T05:31:23Z)
• Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection [119.93025368028083]
  We design a novel Transformer-style Human-Object Interaction (HOI) detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP) STIP decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the non-parametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI
  arXiv  Detail & Related papers  (2022-06-13T16:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.