QUTest: A Native Testing Framework for Quantum Programs

• URL: http://arxiv.org/abs/2605.19736v1
• Date: Tue, 19 May 2026 12:09:14 GMT
• Title: QUTest: A Native Testing Framework for Quantum Programs
• Abstract summary: Quantum programs are often shared as OpenQASM 3 circuits, but tests are still written in host languages such as Python with Qiskit.<n>We present QUTest, a native framework in which both programs and tests are standard.qasm files.<n>Tests follow the Arrange / Act / Assert pattern, while configuration, runtime requirements, and assertions are encoded as pragma comments.
• Score: 4.034061016269666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum programs are often shared as OpenQASM 3 circuits, but tests are still written in host languages such as Python with Qiskit. We present QUTest, a native framework in which both programs and tests are standard .qasm files. Tests follow the Arrange / Act / Assert pattern, while configuration, runtime requirements, and assertions are encoded as pragma comments (//%), preserving compatibility with existing OpenQASM tools. QUTest provides 12 assertion types spanning deterministic, statistical, quantum-state, and structural checks, plus a linter and an environment-aware mode for running the same test across selected runtime versions in isolated environments. Its CLI supports automatic test discovery, runtime compatibility checks, and XML reports for continuous integration. We describe the pragma language, implementation, and a planned evaluation using coverage and mutation testing. QUTest is available at https://github.com/QBugs/qutest. Video demo: https://youtu.be/FvgvsiAXuW0.

Related papers

• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing [57.7131457251794]
  We introduce WebTestBench, a benchmark for evaluating end-to-end automated web testing.<n>We decompose the testing process into two cascaded sub-tasks, checklist generation and defect detection, and propose WebTester.<n>Our findings expose a substantial gap between current computer-use agent capabilities and industrial-grade deployment demands.
  arXiv  Detail & Related papers  (2026-03-26T09:27:29Z)
• Protocol Testing with I/O Grammars [45.68497486907946]
  We propose a novel approach to protocol testing that combines input generation and output checking in a single framework.<n>We demonstrate that I/O grammars can specify advanced protocol features correctly and completely, while also enabling output validation of the programs under test.
  arXiv  Detail & Related papers  (2025-09-24T16:41:04Z)
• Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol [83.83217247686402]
  Large Language Models (LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions.<n>Their inherent non-determinism, dynamism, and context dependence pose fundamental challenges for quality assurance.<n>This paper decomposes LLM applications into a three-layer architecture: textbftextitSystem Shell Layer, textbftextitPrompt Orchestration Layer, and textbftextitLLM Inference Core.
  arXiv  Detail & Related papers  (2025-08-28T13:00:28Z)
• Unify and Triumph: Polyglot, Diverse, and Self-Consistent Generation of Unit Tests with LLMs [5.366354612549172]
  PolyTest is a novel approach that enhances test generation by exploiting polyglot and temperature-controlled diversity.<n>PolyTest mitigates inconsistencies by unifying test sets, fostering self-consistency and improving overall test quality.
  arXiv  Detail & Related papers  (2025-03-20T13:47:06Z)
• ProjectTest: A Project-level LLM Unit Test Generation Benchmark and Impact of Error Fixing Mechanisms [48.43237545197775]
  Unit test generation has become a promising and important use case of LLMs.<n>ProjectTest is a project-level benchmark for unit test generation covering Python, Java, and JavaScript.
  arXiv  Detail & Related papers  (2025-02-10T15:24:30Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch.<n>Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests.<n> Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• ASTER: Natural and Multi-language Unit Test Generation with LLMs [6.259245181881262]
  We describe a generic pipeline that incorporates static analysis to guide LLMs in generating compilable and high-coverage test cases.<n>We conduct an empirical study to assess the quality of the generated tests in terms of code coverage and test naturalness.
  arXiv  Detail & Related papers  (2024-09-04T21:46:18Z)
• TESTEVAL: Benchmarking Large Language Models for Test Case Generation [15.343859279282848]
  We propose TESTEVAL, a novel benchmark for test case generation with large language models (LLMs)<n>We collect 210 Python programs from an online programming platform, LeetCode, and design three different tasks: overall coverage, targeted line/branch coverage, and targeted path coverage.<n>We find that generating test cases to cover specific program lines/branches/paths is still challenging for current LLMs.
  arXiv  Detail & Related papers  (2024-06-06T22:07:50Z)
• TestSpark: IntelliJ IDEA's Ultimate Test Generation Companion [15.13443954421825]
  This paper introduces TestSpark, a plugin for IntelliJ IDEA that enables users to generate unit tests with only a few clicks. TestSpark also allows users to easily modify and run each generated test and integrate them into the project workflow.
  arXiv  Detail & Related papers  (2024-01-12T13:53:57Z)
• A Practical Toolkit for Multilingual Question and Answer Generation [79.31199020420827]
  We introduce AutoQG, an online service for multilingual QAG, along with lmqg, an all-in-one Python package for model fine-tuning, generation, and evaluation. We also release QAG models in eight languages fine-tuned on a few variants of pre-trained encoder-decoder language models, which can be used online via AutoQG or locally via lmqg.
  arXiv  Detail & Related papers  (2023-05-27T08:42:37Z)
• pytest-inline: An Inline Testing Tool for Python [10.307253336106053]
  pytest-inline is a plugin for pytest, the most popular Python testing framework. pytest-inline runs each inline test and fails if the target statement's output does not match the expected output. pytest-inline is integrated into the pytest-dev organization.
  arXiv  Detail & Related papers  (2023-05-22T20:58:44Z)
• ChatUniTest: A Framework for LLM-Based Test Generation [17.296369651892228]
  This paper presents ChatUniTest, an automated unit test generation framework. ChatUniTest incorporates an adaptive focal context mechanism to encompass valuable context in prompts. Our effectiveness evaluation reveals that ChatUniTest outperforms TestSpark and EvoSuite in half of the evaluated projects.
  arXiv  Detail & Related papers  (2023-05-08T15:12:07Z)
• DUAL: Textless Spoken Question Answering with Speech Discrete Unit Adaptive Learning [66.71308154398176]
  Spoken Question Answering (SQA) has gained research attention and made remarkable progress in recent years. Existing SQA methods rely on Automatic Speech Recognition (ASR) transcripts, which are time and cost-prohibitive to collect. This work proposes an ASR transcript-free SQA framework named Discrete Unit Adaptive Learning (DUAL), which leverages unlabeled data for pre-training and is fine-tuned by the SQA downstream task.
  arXiv  Detail & Related papers  (2022-03-09T17:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.