A Methodological Analysis of Empirical Studies in Quantum Software Testing

• URL: http://arxiv.org/abs/2601.08367v1
• Date: Tue, 13 Jan 2026 09:29:00 GMT
• Title: A Methodological Analysis of Empirical Studies in Quantum Software Testing
• Abstract summary: In quantum software engineering (QSE), quantum software testing (QST) has attracted increasing attention as quantum software systems grow in scale and complexity.<n>Since QST evaluates quantum programs through execution under designed test inputs, empirical studies are widely used to assess the effectiveness of testing approaches.<n>This paper presents a methodological analysis of empirical studies in QST through a systematic examination of 59 primary studies identified from a literature pool of size 384.
• Score: 4.872319569157807
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In quantum software engineering (QSE), quantum software testing (QST) has attracted increasing attention as quantum software systems grow in scale and complexity. Since QST evaluates quantum programs through execution under designed test inputs, empirical studies are widely used to assess the effectiveness of testing approaches. However, the design and reporting of empirical studies in QST remain highly diverse, and a shared methodological understanding has yet to emerge, making it difficult to interpret results and compare findings across studies. This paper presents a methodological analysis of empirical studies in QST through a systematic examination of 59 primary studies identified from a literature pool of size 384. We organize our analysis around ten research questions that cover key methodological dimensions of QST empirical studies, including objects under test, baseline comparison, testing setup, experimental configuration, and tool and artifact support. Through cross-study analysis along these dimensions, we characterize current empirical practices in QST, identify recurring limitations and inconsistencies, and highlight open methodological challenges. Based on our findings, we derive insights and recommendations to inform the design, execution, and reporting of future empirical studies in QST.

Related papers

• Exploring Quantum Software Testing Across Research and Practice: Emerging Results from a Multivocal Literature Review [2.012425476229879]
  Our study integrated peer-reviewed studies with gray literature, including blogs, tutorials, forums, technical reports, documentation pages, and company webpages.<n>Our results indicate a rapidly evolving but fragmented ecosystem involving classical adapted testing approaches, quantum-specific techniques, statistical validation methods, simulators, debug environments, and verification frameworks.
  arXiv  Detail & Related papers  (2026-08-24T02:39:24Z)
• Benchmarking Quantum Software Testing with Scalable Quantum Programs [9.3695909584334]
  We present Qolumbina, a benchmark infrastructure for controlled QST experiments on scalable quantum programs.<n>Qlumbina curates 40 programs from open-source repositories, turns them into test-ready subjects.<n>We also propose QST-oriented criteria to characterize quantum programs along functionality, output behavior, development complexity, and quantum-specific execution complexity.
  arXiv  Detail & Related papers  (2026-07-02T10:56:35Z)
• A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era [57.5440098776759]
  Molecular property prediction integrates quantum chemistry, cheminformatics, and deep learning to connect molecular structure with biological behavior.<n>This survey traces four complementary paradigms, including Quantum, Descriptor Machine Learning, Geometric Deep Learning, and Foundation Models.<n> Benchmark analyses integrate evidence from both widely used datasets and datasets reflecting industry perspectives.
  arXiv  Detail & Related papers  (2026-04-17T15:16:33Z)
• Validation of an analyzability model for quantum software: a family of experiments [3.4234775021463193]
  The analyzability of hybrid software, which integrates both classical and quantum components, is a key factor in ensuring its maintainability and industrial adoption.<n>This article presents the empirical validation, through a family of experiments, of the quantum component of a previously proposed hybrid software analyzability model.
  arXiv  Detail & Related papers  (2026-02-24T16:35:20Z)
• Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network [46.56867772369597]
  Best practices for meta-algorithmic research exist, but they are scattered between different publications and fields.<n>This report collects good practices for empirical meta-algorithmic research across the subfields of the COSEAL community.<n>It establishes the current state-of-the-art practices within meta-algorithmic research and serves as a guideline to both new researchers and practitioners in meta-algorithmic fields.
  arXiv  Detail & Related papers  (2025-12-18T12:59:45Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• Empirical Studies on Quantum Optimization for Software Engineering: A Systematic Analysis [8.44997508165897]
  Quantum, quantum-inspired, and hybrid algorithms are increasingly showing promise for solving software engineering optimization problems.<n>Best-intended practices for conducting empirical studies have not yet well established.
  arXiv  Detail & Related papers  (2025-10-31T02:28:11Z)
• Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications [59.721265428780946]
  Large Language Models (LLMs) in medicine have enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning.<n>This paper provides the first systematic review of this emerging field.<n>We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies and test-time mechanisms.
  arXiv  Detail & Related papers  (2025-08-01T14:41:31Z)
• AI-Driven Tools in Modern Software Quality Assurance: An Assessment of Benefits, Challenges, and Future Directions [0.0]
  The research aims to assess the benefits, challenges, and prospects of integrating modern AI-oriented tools into quality assurance processes.<n>The research demonstrates AI's transformative potential for QA but highlights the importance of a strategic approach to implementing these technologies.
  arXiv  Detail & Related papers  (2025-06-19T20:22:47Z)
• A Call for Critically Rethinking and Reforming Data Analysis in Empirical Software Engineering [5.687882380471718]
  Concerns about the correct application of empirical methodologies have existed since the 2006 Dagstuhl seminar on Empirical Software Engineering.<n>We conducted a literature survey of 27,000 empirical studies, using LLMs to classify statistical methodologies as adequate or inadequate.<n>We selected 30 primary studies and held a workshop with 33 ESE experts to assess their ability to identify and resolve statistical issues.
  arXiv  Detail & Related papers  (2025-01-22T09:05:01Z)
• Survey of Computerized Adaptive Testing: A Machine Learning Perspective [66.26687542572974]
  Computerized Adaptive Testing (CAT) provides an efficient and tailored method for assessing the proficiency of examinees. This paper aims to provide a machine learning-focused survey on CAT, presenting a fresh perspective on this adaptive testing method.
  arXiv  Detail & Related papers  (2024-03-31T15:09:47Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Unifying (Quantum) Statistical and Parametrized (Quantum) Algorithms [65.268245109828]
  We take inspiration from Kearns' SQ oracle and Valiant's weak evaluation oracle. We introduce an extensive yet intuitive framework that yields unconditional lower bounds for learning from evaluation queries.
  arXiv  Detail & Related papers  (2023-10-26T18:23:21Z)
• Testing Multi-Subroutine Quantum Programs: From Unit Testing to Integration Testing [2.8611507672161265]
  This paper addresses the specific testing requirements of multi-subroutine quantum programs. We focus on testing criteria and techniques based on the whole testing process perspective. We conduct comprehensive testing on typical quantum subroutines, including diverse mutants and randomized inputs.
  arXiv  Detail & Related papers  (2023-06-30T05:31:56Z)
• Phase Transition Behavior in Knowledge Compilation [52.68422776053012]
  We study the behaviour of size and compile-time behaviour for random k-CNF formulas in the context of knowledge compilation. Our work is similar in spirit to the early work in CSP community on phase transition behavior in SAT/CSP.
  arXiv  Detail & Related papers  (2020-07-20T18:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.