QUARK: A Framework for Quantum Computing Application Benchmarking

• URL: http://arxiv.org/abs/2202.03028v3
• Date: Fri, 5 Aug 2022 12:15:57 GMT
• Title: QUARK: A Framework for Quantum Computing Application Benchmarking
• Authors: Jernej Rudi Fin\v{z}gar, Philipp Ross, Leonhard H\"olscher, Johannes Klepsch, Andre Luckow
• Abstract summary: We propose an application-centric benchmark method and the QUARK framework to foster the investigation and creation of application benchmarks for QC. This paper makes a case for application-level benchmarks and provides an in-depth "pen and paper" benchmark formulation of two reference problems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum computing (QC) is anticipated to provide a speedup over classical HPC approaches for specific problems in optimization, simulation, and machine learning. With the advances in quantum computing toward practical applications, the need to analyze and compare different quantum solutions increases. While different low-level benchmarks for QC exist, these benchmarks do not provide sufficient insights into real-world application-level performance. We propose an application-centric benchmark method and the QUantum computing Application benchmaRK (QUARK) framework to foster the investigation and creation of application benchmarks for QC. This paper establishes three significant contributions: (1) it makes a case for application-level benchmarks and provides an in-depth "pen and paper" benchmark formulation of two reference problems: robot path and vehicle option optimization from the industrial domain; (2) it proposes the open-source QUARK framework for designing, implementing, executing, and analyzing benchmarks; (3) it provides multiple reference implementations for these two reference problems based on different known, and where needed, extended, classical and quantum algorithmic approaches and analyzes their performance on different types of infrastructures.

Related papers

• BenchQC -- Scalable and modular benchmarking of industrial quantum computing applications [26.629709879735532]
  BenchQC promotes an application-centric perspective for benchmarking real-world quantum applications. We aim to uncover meaningful trends, provide systematic guidance on quantum utility, and distinguish promising research directions from less viable approaches.
  arXiv  Detail & Related papers  (2025-04-15T14:05:11Z)
• Systematic benchmarking of quantum computers: status and recommendations [1.1961811541956795]
  Benchmarking is crucial for assessing the performance of quantum computers. The document highlights key aspects such as component-level, system-level, software-level, HPC-level, and application-level benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T19:05:13Z)
• Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis [89.60263788590893]
  Post-training Quantization (PTQ) technique has been extensively adopted for large language models (LLMs) compression. Existing algorithms focus primarily on performance, overlooking the trade-off among model size, performance, and quantization bitwidth. We provide a novel benchmark for LLMs PTQ in this paper.
  arXiv  Detail & Related papers  (2025-02-18T07:35:35Z)
• Towards Robust Benchmarking of Quantum Optimization Algorithms [3.9456729020535013]
  A key problem in existing benchmarking frameworks is the lack of equal effort in optimizing for the best quantum and, respectively, classical approaches. This paper presents a comprehensive set of guidelines comprising universal steps towards fair benchmarks.
  arXiv  Detail & Related papers  (2024-05-13T10:35:23Z)
• HamilToniQ: An Open-Source Benchmark Toolkit for Quantum Computers [4.795321943127061]
  HamilToniQ is an open-source, application-oriented benchmarking toolkit for the comprehensive evaluation of Quantum Processing Units (QPUs) It incorporates a methodological framework assessing QPU types, topologies, and multi-QPU systems. HamilToniQ's standardized score, H-Score, quantifies the fidelity and reliability of QPUs.
  arXiv  Detail & Related papers  (2024-04-22T08:27:14Z)
• Quantum Computing Enhanced Service Ecosystem for Simulation in Manufacturing [56.61654656648898]
  We propose a framework for a quantum computing-enhanced service ecosystem for simulation in manufacturing. We analyse two high-value use cases with the aim of a quantitative evaluation of these new computing paradigms for industrially-relevant settings.
  arXiv  Detail & Related papers  (2024-01-19T11:04:14Z)
• Qubit efficient quantum algorithms for the vehicle routing problem on NISQ processors [48.68474702382697]
  Vehicle routing problem with time windows (VRPTW) is a common optimization problem faced within the logistics industry. In this work, we explore the use of a previously-introduced qubit encoding scheme to reduce the number of binary variables.
  arXiv  Detail & Related papers  (2023-06-14T13:44:35Z)
• A Framework for Demonstrating Practical Quantum Advantage: Racing Quantum against Classical Generative Models [62.997667081978825]
  We build over a proposed framework for evaluating the generalization performance of generative models. We establish the first comparative race towards practical quantum advantage (PQA) between classical and quantum generative models. Our results suggest that QCBMs are more efficient in the data-limited regime than the other state-of-the-art classical generative models.
  arXiv  Detail & Related papers  (2023-03-27T22:48:28Z)
• End-to-end resource analysis for quantum interior point methods and portfolio optimization [63.4863637315163]
  We provide a complete quantum circuit-level description of the algorithm from problem input to problem output. We report the number of logical qubits and the quantity/depth of non-Clifford T-gates needed to run the algorithm.
  arXiv  Detail & Related papers  (2022-11-22T18:54:48Z)
• QPack Scores: Quantitative performance metrics for application-oriented quantum computer benchmarking [1.0323063834827415]
  This paper presents the benchmark score definitions of QPack, an application-oriented cross-platform benchmarking suite for quantum computers and simulators. A comparison is made between various quantum computer simulators, running both locally and on vendors' remote cloud services.
  arXiv  Detail & Related papers  (2022-05-24T15:18:24Z)
• Quantum circuit architecture search on a superconducting processor [56.04169357427682]
  Variational quantum algorithms (VQAs) have shown strong evidences to gain provable computational advantages for diverse fields such as finance, machine learning, and chemistry. However, the ansatz exploited in modern VQAs is incapable of balancing the tradeoff between expressivity and trainability. We demonstrate the first proof-of-principle experiment of applying an efficient automatic ansatz design technique to enhance VQAs on an 8-qubit superconducting quantum processor.
  arXiv  Detail & Related papers  (2022-01-04T01:53:42Z)
• Accelerating variational quantum algorithms with multiple quantum processors [78.36566711543476]
  Variational quantum algorithms (VQAs) have the potential of utilizing near-term quantum machines to gain certain computational advantages. Modern VQAs suffer from cumbersome computational overhead, hampered by the tradition of employing a solitary quantum processor to handle large data. Here we devise an efficient distributed optimization scheme, called QUDIO, to address this issue.
  arXiv  Detail & Related papers  (2021-06-24T08:18:42Z)
• QPack: Quantum Approximate Optimization Algorithms as universal benchmark for quantum computers [1.1602089225841632]
  We present QPack, a universal benchmark for Noisy Intermediate-Scale Quantum (NISQ) computers. QPack evaluates the maximum problem size a quantum computer can solve, the required runtime, as well as the achieved accuracy.
  arXiv  Detail & Related papers  (2021-03-31T16:20:51Z)
• Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack [0.0]
  Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. We propose 3 "application-motivated" circuit classes for benchmarking: deep, shallow, and square. We quantify the performance of quantum computing system in running circuits from these classes using several figures of merit.
  arXiv  Detail & Related papers  (2020-06-01T21:21:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.