QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging

• URL: http://arxiv.org/abs/2606.07314v1
• Date: Fri, 05 Jun 2026 14:34:46 GMT
• Title: QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging
• Abstract summary: Large language models (LLMs) have shown strong performance on classical software engineering tasks.<n>We propose QBugLM, a multi-agent framework that automates the quantum software debug pipeline.
• Score: 0.8609132348927196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Quantum software bugs often yield silent, incorrect outputs rather than explicit errors, making them particularly difficult to detect and repair with conventional techniques. Although large language models (LLMs) have shown strong performance on classical software engineering tasks, their ability to debug quantum code remains largely unexplored. To bridge this gap, we propose QBugLM, a multi-agent framework that automates the quantum software debugging pipeline, from taxonomy-driven bug injection to LLM-based detection and repair, and finally to simulation-based validation, for framework-agnostic OpenQASM 3.0 programs. We further conduct a comprehensive case study using QBugLM to benchmark two LLMs, Claude 4.6 Sonnet and Qwen3 Coder Next, across different prompting strategies, bug categories, and quantum programs. Our results show that iterative feedback is critical, as a single retry raises Pass@1 from below 25% to above 80%. Moreover, simpler structured prompting can even outperform Chain-of-Thought and ReAct for reasoning-capable models under fixed-resource constraints. Our work takes initial steps toward benchmarking LLM capabilities for debugging quantum programs and offers practical insights to support future efforts in automated quantum software repair.

Related papers

• Qiskit Code Migration with LLMs [0.13683072934528265]
  This paper proposes a hybrid approach integrating Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) to automate the migration of Qiskit code across versions.<n>The proposed methodology enhances the precision and reliability of migration suggestions by leveraging an automatically generated taxonomy of migration scenarios.
  arXiv  Detail & Related papers  (2026-06-18T12:40:43Z)
• Beyond Rules: LLM-Powered Linting for Quantum Programs [4.968531828144473]
  We introduce LintQ-LLM+CoT and LintQ-LLM+RAG, novel approaches that redefine the detection of quantum programming problems.<n>LLMs specialized via Chain-of-Thought (CoT) prompting and a Retrieval-Augmented Generation (RAG) system grounds the model's reasoning in a curated knowledge base.<n>Our results show that LLM-based approaches, with and without RAG, outperform LintQ in terms of quantum programming problems detection correctness (precision) and completeness (recall)
  arXiv  Detail & Related papers  (2026-05-05T16:31:14Z)
• QuanBench: Benchmarking Quantum Code Generation with Large Language Models [7.807551490308163]
  Large language models (LLMs) have demonstrated good performance in general code generation.<n>This paper presents QuanBench, a benchmark for evaluating LLMs on quantum code generation.
  arXiv  Detail & Related papers  (2025-10-19T10:08:36Z)
• QAgent: An LLM-based Multi-Agent System for Autonomous OpenQASM programming [8.73473101831257]
  We present QAgent, a multi-agent system that fully automates OpenQASM programming.<n>Our evaluations demonstrate substantial improvements across multiple LLMs of varying sizes.<n>We envision this multi-agent system as a key enabler for democratizing quantum programming, bridging expertise gaps, and accelerating the practical adoption of quantum computing.
  arXiv  Detail & Related papers  (2025-08-26T18:40:02Z)
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs [78.09559830840595]
  We present the first systematic study on quantizing diffusion-based language models.<n>We identify the presence of activation outliers, characterized by abnormally large activation values.<n>We implement state-of-the-art PTQ methods and conduct a comprehensive evaluation.
  arXiv  Detail & Related papers  (2025-08-20T17:59:51Z)
• MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision [76.42361936804313]
  We introduce MAS-ZERO, the first self-evolved, inference-time framework for automatic MAS design.<n> MAS-ZERO employs meta-level design to iteratively generate, evaluate, and refine MAS configurations tailored to each problem instance.
  arXiv  Detail & Related papers  (2025-05-21T00:56:09Z)
• Agent-Q: Fine-Tuning Large Language Models for Quantum Circuit Generation and Optimization [5.2374261493530065]
  Large language models (LLMs) have achieved remarkable outcomes in complex problems, including math, coding, and analyzing scientific reports.<n>We describe Agent-Q, an LLM fine-tuning system to generate and optimize quantum circuits.
  arXiv  Detail & Related papers  (2025-04-15T11:56:54Z)
• Quantum Program Linting with LLMs: Emerging Results from a Comparative Study [5.062046608347911]
  This study investigates the feasibility of employing Large Language Models (LLMs) to develop a novel linting technique for quantum software development.<n>We introduce LintQ-LLM, an LLM-based linting tool designed to detect quantum-specific problems comparable to those identified by LintQ.
  arXiv  Detail & Related papers  (2025-04-07T15:51:31Z)
• QSpec: Speculative Decoding with Complementary Quantization Schemes [53.960146187821685]
  Quantization is widely adopted to accelerate inference and reduce memory consumption in large language models (LLMs)<n>We propose QSpec, a novel quantization paradigm that decouples efficiency from quality.<n>QSpec reuses both weights and KV cache across stages, enabling near-zero-cost switching without retraining or auxiliary models.
  arXiv  Detail & Related papers  (2024-10-15T05:57:51Z)
• QCircuitBench: A Large-Scale Dataset for Benchmarking Quantum Algorithm Design [63.02824918725805]
  Quantum computing is recognized for the significant speedup it offers over classical computing through quantum algorithms.<n>QCircuitBench is the first benchmark dataset designed to evaluate AI's capability in designing and implementing quantum algorithms.
  arXiv  Detail & Related papers  (2024-10-10T14:24:30Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification is a key element of machine learning applications.<n>We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines.<n>We conduct a large-scale empirical investigation of UQ and normalization techniques across eleven tasks, identifying the most effective approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)
• LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [55.73370804397226]
  Quantization, a key compression technique, can effectively mitigate these demands by compressing and accelerating large language models. We present LLMC, a plug-and-play compression toolkit, to fairly and systematically explore the impact of quantization. Powered by this versatile toolkit, our benchmark covers three key aspects: calibration data, algorithms (three strategies), and data formats.
  arXiv  Detail & Related papers  (2024-05-09T11:49:05Z)
• Sampling Overhead Analysis of Quantum Error Mitigation: Uncoded vs. Coded Systems [69.33243249411113]
  We show that Pauli errors incur the lowest sampling overhead among a large class of realistic quantum channels. We conceive a scheme amalgamating QEM with quantum channel coding, and analyse its sampling overhead reduction compared to pure QEM.
  arXiv  Detail & Related papers  (2020-12-15T15:51:27Z)
• Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
  We propose a resource and runtime efficient scheme termed quantum architecture search (QAS) QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates. We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
  arXiv  Detail & Related papers  (2020-10-20T12:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.