Benchmarking Quantum Computers: Towards a Standard Performance Evaluation Approach

• URL: http://arxiv.org/abs/2407.10941v2
• Date: Wed, 17 Jul 2024 16:27:14 GMT
• Title: Benchmarking Quantum Computers: Towards a Standard Performance Evaluation Approach
• Authors: Arturo Acuaviva, David Aguirre, Rubén Peña, Mikel Sanz,
• Abstract summary: We review the most important aspects of both classical processor benchmarks and the metrics comprising them. We analyze the intrinsic properties that characterize the paradigm of quantum computing. We propose general guidelines for quantum benchmarking.
• Score: 0.7499722271664147
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The technological development of increasingly larger quantum processors on different quantum platforms raises the problem of how to fairly compare their performance, known as quantum benchmarking of quantum processors. This is a challenge that computer scientists have already faced when comparing classical processors, leading to the development of various mathematical tools to address it, but also to the identification of the limits of this problem. In this work, we briefly review the most important aspects of both classical processor benchmarks and the metrics comprising them, providing precise definitions and analyzing the quality attributes that they should exhibit. Subsequently, we analyze the intrinsic properties that characterize the paradigm of quantum computing and hinder the naive transfer of strategies from classical benchmarking. However, we can still leverage some of the lessons learned such as the quality attributes of a \textit{good} benchmark. Additionally, we review some of the most important metrics and benchmarks for quantum processors proposed in the literature, assessing what quality attributes they fulfill. Finally, we propose general guidelines for quantum benchmarking. These guidelines aim to pave the way for establishing a roadmap towards standardizing the performance evaluation of quantum devices, ultimately leading to the creation of an organization akin to the Standard Performance Evaluation Corporation (SPEC).

Related papers

• Benchmarking quantum machine learning kernel training for classification tasks [0.0]
  This study focuses on quantum kernel methods in the context of classification tasks. It examines the performance of Quantum Kernel Estimation (QKE) and Quantum Kernel Training (QKT) in connection with two quantum feature mappings. Experimental results indicate that quantum methods exhibit varying performance across different datasets.
  arXiv  Detail & Related papers  (2024-08-17T10:53:06Z)
• Benchmarking quantum computers [0.0]
  Good benchmarks empower scientists, engineers, programmers, and users to understand a computing system's power. Bad benchmarks can misdirect research and inhibit progress. We discuss the role of benchmarks and benchmarking, and how good benchmarks can drive and measure progress.
  arXiv  Detail & Related papers  (2024-07-11T19:25:30Z)
• Randomized Benchmarking of Local Zeroth-Order Optimizers for Variational Quantum Systems [65.268245109828]
  We compare the performance of classicals across a series of partially-randomized tasks. We focus on local zeroth-orders due to their generally favorable performance and query-efficiency on quantum systems.
  arXiv  Detail & Related papers  (2023-10-14T02:13:26Z)
• Majorization-based benchmark of the complexity of quantum processors [105.54048699217668]
  We numerically simulate and characterize the operation of various quantum processors. We identify and assess quantum complexity by comparing the performance of each device against benchmark lines. We find that the majorization-based benchmark holds as long as the circuits' output states have, on average, high purity.
  arXiv  Detail & Related papers  (2023-04-10T23:01:10Z)
• Quantum utility -- definition and assessment of a practical quantum advantage [0.0]
  Different use-cases come with different requirements for size, weight, power consumption, or data privacy. This paper aims to incorporate these characteristics into a concept coined quantum utility. It demonstrates the effectiveness and practicality of quantum computers for various applications.
  arXiv  Detail & Related papers  (2023-03-03T18:33:46Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• SupermarQ: A Scalable Quantum Benchmark Suite [3.6806897290408305]
  SupermarQ is a scalable, hardware-agnostic quantum benchmark suite which uses application-level metrics to measure performance. SupermarQ is the first attempt to systematically apply techniques from classical benchmarking methodology to the quantum domain. We envision that quantum benchmarking will encompass a large cross-community effort built on open source, constantly evolving benchmark suites.
  arXiv  Detail & Related papers  (2022-02-22T17:24:07Z)
• Generalization Metrics for Practical Quantum Advantage in Generative Models [68.8204255655161]
  Generative modeling is a widely accepted natural use case for quantum computers. We construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Our simulation results show that our quantum-inspired models have up to a $68 times$ enhancement in generating unseen unique and valid samples.
  arXiv  Detail & Related papers  (2022-01-21T16:35:35Z)
• Quantum circuit architecture search on a superconducting processor [56.04169357427682]
  Variational quantum algorithms (VQAs) have shown strong evidences to gain provable computational advantages for diverse fields such as finance, machine learning, and chemistry. However, the ansatz exploited in modern VQAs is incapable of balancing the tradeoff between expressivity and trainability. We demonstrate the first proof-of-principle experiment of applying an efficient automatic ansatz design technique to enhance VQAs on an 8-qubit superconducting quantum processor.
  arXiv  Detail & Related papers  (2022-01-04T01:53:42Z)
• Application-Oriented Performance Benchmarks for Quantum Computing [0.0]
  benchmarking suite is designed to be readily accessible to a broad audience of users. Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years.
  arXiv  Detail & Related papers  (2021-10-07T01:45:06Z)
• Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
  We propose a resource and runtime efficient scheme termed quantum architecture search (QAS) QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates. We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
  arXiv  Detail & Related papers  (2020-10-20T12:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.