Ising-based Test Optimization and Benchmarking

• URL: http://arxiv.org/abs/2604.10450v1
• Date: Sun, 12 Apr 2026 04:20:23 GMT
• Title: Ising-based Test Optimization and Benchmarking
• Abstract summary: We present IsingTester, a command-line tool for solving test optimization problems.<n>We reformulate test selection and minimization as Ising spin configurations, encode multiple optimization strategies into Ising Hamiltonians, and implement solvers including CIM simulation and brute-force search.
• Score: 4.85142635110053
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test optimization contains test case selection and minimization, which is an important challenge in software testing and has been addressed with search-based approaches intensively in the past. Inspired by the recent advancement of using quantum optimization solutions for addressing test optimization problems, we looked into Coherent Ising Machines (CIM), which offer potential for solving combinatorial optimization problems, but have not yet been exploited in test optimization. Hence, in this paper, we present IsingTester, an open-source, Python-based command-line tool that provides an end-to-end pipeline for solving test optimization problems that are formulated as Ising models. With IsingTester, we reformulate test selection and minimization as Ising spin configurations, encode multiple optimization strategies into Ising Hamiltonians, and implement solvers including CIM simulation and brute-force search. Given a user-provided dataset and solver configuration, IsingTester automatically performs problem encoding, optimization, and spin decoding, returning selected test cases back to the user. Along with IsingTester, we also present the accompanying IsingBench for evaluating and comparing optimization techniques across Ising-based paradigms against baseline approaches. A screencast demonstrating the tool is available at: https://github.com/WSE-Lab/IsingBench.

Related papers

• POLCA: Stochastic Generative Optimization with LLM [26.342554900977003]
  We introduce Prioritized Optimization with Local Contextualion (POLCA)<n>POLCA is a scalable framework designed to handle parameterity in optimization.<n>We demonstrate POLCA achieves robust, sample and time-efficient performance.
  arXiv  Detail & Related papers  (2026-03-16T03:07:44Z)
• Sample, Don't Search: Rethinking Test-Time Alignment for Language Models [55.2480439325792]
  We introduce QAlign, a new test-time alignment approach.<n>As we scale test-time compute, QAlign converges to sampling from the optimal aligned distribution for each individual prompt.<n>By adopting recent advances in Markov chain Monte Carlo for text generation, our method enables better-aligned outputs without modifying the underlying model or even requiring logit access.
  arXiv  Detail & Related papers  (2025-04-04T00:41:40Z)
• Learning Multiple Initial Solutions to Optimization Problems [52.9380464408756]
  Sequentially solving similar optimization problems under strict runtime constraints is essential for many applications.<n>We propose learning to predict emphmultiple diverse initial solutions given parameters that define the problem instance.<n>We find significant and consistent improvement with our method across all evaluation settings and demonstrate that it efficiently scales with the number of initial solutions required.
  arXiv  Detail & Related papers  (2024-11-04T15:17:19Z)
• AutoTest: Evolutionary Code Solution Selection with Test Cases [1.4582633500696451]
  This study proposes AutoTest, a novel technique that combines automated test case generation with code solution execution. The HumanEval dataset consists of 164 programming problems, and AutoTest achieves approximately a 10% improvement over the baseline method in terms of pass@1 score.
  arXiv  Detail & Related papers  (2024-08-22T04:38:41Z)
• OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling [62.19438812624467]
  Large language models (LLMs) have exhibited their problem-solving abilities in mathematical reasoning.<n>We propose OptiBench, a benchmark for End-to-end optimization problem-solving with human-readable inputs and outputs.
  arXiv  Detail & Related papers  (2024-07-13T13:27:57Z)
• CATBench: A Compiler Autotuning Benchmarking Suite for Black-box Optimization [5.909352339240516]
  We present CATBench, a comprehensive benchmarking suite that captures the complexities of compiler autotuning.<n>The benchmarks in CATBench span a range of machine learning-oriented computations, from tensor algebra to image processing and clustering.<n>We validate CATBench on several state-of-the-art algorithms, revealing their strengths and weaknesses.
  arXiv  Detail & Related papers  (2024-06-24T20:15:04Z)
• On Test Sequence Generation using Multi-Objective Particle Swarm Optimization [0.2999888908665658]
  Software testing is an important and essential part of the software development life cycle. In the software industry, testing costs can account for about 35% to 40% of the total cost of a software project.
  arXiv  Detail & Related papers  (2024-04-09T18:35:21Z)
• JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning [58.71541261221863]
  Join order selection (JOS) is the problem of ordering join operations to minimize total query execution cost. We present JoinGym, a query optimization environment for bushy reinforcement learning (RL) Under the hood, JoinGym simulates a query plan's cost by looking up intermediate result cardinalities from a pre-computed dataset.
  arXiv  Detail & Related papers  (2023-07-21T17:00:06Z)
• An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization [78.36413169647408]
  We study the effectiveness of various ZO optimization methods for optimizing molecular objectives. We show the advantages of ZO sign-based gradient descent (ZO-signGD) We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
  arXiv  Detail & Related papers  (2022-10-27T01:58:10Z)
• Efficient Non-Parametric Optimizer Search for Diverse Tasks [93.64739408827604]
  We present the first efficient scalable and general framework that can directly search on the tasks of interest. Inspired by the innate tree structure of the underlying math expressions, we re-arrange the spaces into a super-tree. We adopt an adaptation of the Monte Carlo method to tree search, equipped with rejection sampling and equivalent- form detection.
  arXiv  Detail & Related papers  (2022-09-27T17:51:31Z)
• Optimizer Amalgamation [124.33523126363728]
  We are motivated to study a new problem named Amalgamation: how can we best combine a pool of "teacher" amalgamations into a single "student" that can have stronger problem-specific performance? First, we define three differentiable mechanisms to amalgamate a pool of analyticals by gradient descent. In order to reduce variance of the process, we also explore methods to stabilize the process by perturbing the target.
  arXiv  Detail & Related papers  (2022-03-12T16:07:57Z)
• Bayesian Optimization for auto-tuning GPU kernels [0.0]
  Finding optimal parameter configurations for GPU kernels is a non-trivial exercise for large search spaces, even when automated. We introduce a novel contextual exploration factor as well as new acquisition functions with improved scalability, combined with an informed function selection mechanism.
  arXiv  Detail & Related papers  (2021-11-26T11:26:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.