Metal-Sci: A Scientific Compute Benchmark for Evolutionary LLM Kernel Search on Apple Silicon

• URL: http://arxiv.org/abs/2605.09708v1
• Date: Sun, 10 May 2026 19:11:26 GMT
• Title: Metal-Sci: A Scientific Compute Benchmark for Evolutionary LLM Kernel Search on Apple Silicon
• Abstract summary: Metal-Sci is a 10-task benchmark of scientific Apple Silicon Metal compute kernels.<n>Each task ships a CPU reference, a roofline-anchored fitness function, and a held-out generalization size.<n>We report matched single-model sweeps of Claude Opus 4.7, Gemini 3.1 Pro, and GPT 5.5 on M1 Pro.
• Score: 6.599344783327054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in $n$-body problems, multi-field Boltzmann, neighbor-list molecular dynamics, multi-kernel PDE, FFT). Each task ships a CPU reference, a roofline-anchored fitness function, and a held-out generalization size. We pair the benchmark with a lightweight harness for automatic kernel search that runtime-compiles each candidate, scores it against the roofline across multiple sizes, and feeds structured compile and per-size correctness diagnostics back to a frozen LLM driving a $(1{+}1)$ evolutionary loop. We report matched single-model sweeps of Claude Opus 4.7, Gemini 3.1 Pro, and GPT 5.5 on M1 Pro: in-distribution self-speedups span $1.00\times$ to $10.7\times$. Beyond raw speedup, our central methodological claim is structural: the held-out gate scoring function $Φ_\mathcal{T}$ (evaluated once at end-of-run on a configuration the agent never sees during search) functions as a cheap mechanical oversight primitive on this automatic search loop, catching e.g. an Opus template <uint D> HMC win that returns wrong samples at unseen dimensions, and a GPT FFT3D best that wins in-distribution at $2.95\times$ speedup but collapses to $0.23\times$ on a $256^3$ held-out cube, a silent regression that the in-distribution score alone cannot see. Code at https://github.com/vicgalle/metal-sci-kernels

Related papers

• Ockhamareto: Pareto-Gated Segment-Level Credit Assignment for Concise Unit-Test Generation with Reinforcement Learning [64.49364696628294]
  We introduce textbfOckhamareto, a single-shot GRPO framework for unit-test generation and selection.<n>Ockhamareto has two principal components: (i)a emphPareto-gated Bonus that rewards only rollouts non-dominated in(mutation, $-$#tests) space, and (ii)emphToken-level Segment Credit, which attributes each test's marginal mutation kills back to the tokens of its unit-test block.
  arXiv  Detail & Related papers  (2026-08-25T12:20:02Z)
• TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes [6.460910545652521]
  TEMPO is a makespan-aware dispatcher solving per-batch dispatch in milliseconds off the critical path.<n>Its SG integration runs out-of-process and fuses dispatch with count into one in-graph kernel.<n>End-to-end on Testbed B, Qwen3-235B (inside the win region) gains $4$--$6%$ throughput and cuts p99 latency by $sim 15.6%$; DeepSeek-V3 shows only mechanism cost.
  arXiv  Detail & Related papers  (2026-08-13T10:21:11Z)
• Gaming Without an Attacker: Benchmark Fingerprinting in LLM-Driven Search Under Selection Pressure [6.599344783327054]
  Benchmarks for systems that are optimized against the evaluation signal measure something different from what they claim.<n>We document this concretely in two GPU- kernel-optimization suites with held-out generalization gates.<n>We give a four-mode taxonomy of these failures, from configuration fingerprints to gate leakage.
  arXiv  Detail & Related papers  (2026-08-09T14:16:11Z)
• Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling [55.388363730120325]
  We parallelize the mainstream $$-leaping algorithm for absorbing discrete diffusion in a Continuous-Time Markov Chain (CTMC) framework.<n>By leveraging the continuous-time integral form of the $$-leaping algorithm and the Picard method, we achieve parallel-in-time sampling acceleration.<n>Our research expands the potential of discrete diffusion models for efficient parallel inference, with broader implications for applications such as molecular structure and language generation.
  arXiv  Detail & Related papers  (2026-07-01T10:59:33Z)
• Spectral Anatomy of Quantum Gaussian Process Kernels [38.264196157340216]
  We prove a Cauchy-Schwarz tail bound on Nystrm approximation error, a finite-sample variance-contraction identity, and a characterization of the emphtarget-dependent optimal entropy.<n>The diagnostic is kernel-agnostic: hardware-efficient, matchgate, IQP emphand RBF/Matérn/RFF/deep- kernel families all collapse onto identical $S/log n$ curves.
  arXiv  Detail & Related papers  (2026-05-29T07:41:14Z)
• Two Layers, No Swaps: Biplanar SPOQC Architecture Improves Runtime of Fermi-Hubbard Simulation [0.0]
  We estimate the cost of simulating the two-dimensional Fermi-Hubbard model on a biplanar spin-optical quantum computing architecture.<n>We benchmark lattice surgery and magic state preparation within each plane.<n>We find that fallback-based rotation synthesis methods become a scalability bottleneck.
  arXiv  Detail & Related papers  (2026-05-06T18:00:09Z)
• Hardware Validation of DAGI via a Modular "Ridge" Signature and High-Order Synergistic Information [0.0]
  DAGI (Directed Acyclic Graph Information) framework on IBM Quantum hardware.<n>Small, controlled experiment whose ideal output distribution is constrained to a low-dimensional modular manifold (a "ridge)<n>Key recovery exceeds chance: per-shot accuracy 0.1689 (chance 0.125, 95% Wilson CI [0.1610, 0.1772])<n>These results support the claim that DAGI detects and quantifies non-trivial, hardware-resilient information structure.
  arXiv  Detail & Related papers  (2026-04-16T14:16:59Z)
• FHECore: Rethinking GPU Microarchitecture for Fully Homomorphic Encryption [2.7777199166440827]
  Fully Homomorphic Encryption (FHE) enables computation directly on encrypted data but incurs massive computational and memory overheads.<n>Custom accelerators can mitigate these costs, but their long time-to-market and the rapid evolution of FHE algorithms threaten their long-term relevance.<n>We propose FHECore, a specialized functional unit integrated directly into the GPU's Streaming Multiprocessor.
  arXiv  Detail & Related papers  (2026-02-10T02:55:10Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Graph Random Features for Scalable Gaussian Processes [52.89901965157282]
  We study the application of graph random features (GRFs) to scalable Gaussian processes on discrete input spaces.<n>We prove that (under mild assumptions) Bayesian inference with GRFs enjoys $O(N3/2)$ time complexity with respect to the number of nodes $N$, compared to $O(N3)$ for exact kernels.
  arXiv  Detail & Related papers  (2025-09-03T20:13:23Z)
• Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction [57.93371273485736]
  We consider a centralized distributed learning setup where all workers jointly find an unbiased bound LDeltaepsilon2,$ better poly-logarithmically in $n$, even in the homogeneous (i.i.d.) case, where all workers access the same distribution.
  arXiv  Detail & Related papers  (2025-06-30T13:27:39Z)
• Inertial Confinement Fusion Forecasting via Large Language Models [48.76222320245404]
  In this study, we introduce $textbfLPI-LLM$, a novel integration of Large Language Models (LLMs) with classical reservoir computing paradigms. We propose the $textitLLM-anchored Reservoir$, augmented with a $textitFusion-specific Prompt$, enabling accurate forecasting of $textttLPI$-generated-hot electron dynamics during implosion. We also present $textbfLPI4AI$, the first $textttLPI$ benchmark based
  arXiv  Detail & Related papers  (2024-07-15T05:46:44Z)
• Reward-Mixing MDPs with a Few Latent Contexts are Learnable [75.17357040707347]
  We consider episodic reinforcement learning in reward-mixing Markov decision processes (RMMDPs) Our goal is to learn a near-optimal policy that nearly maximizes the $H$ time-step cumulative rewards in such a model.
  arXiv  Detail & Related papers  (2022-10-05T22:52:00Z)
• TURF: A Two-factor, Universal, Robust, Fast Distribution Learning Algorithm [64.13217062232874]
  One of its most powerful and successful modalities approximates every distribution to an $ell$ distance essentially at most a constant times larger than its closest $t$-piece degree-$d_$. We provide a method that estimates this number near-optimally, hence helps approach the best possible approximation.
  arXiv  Detail & Related papers  (2022-02-15T03:49:28Z)
• Giga-scale Kernel Matrix Vector Multiplication on GPU [19.663081364196778]
  Kernel matrix-vector multiplication (KMVM) is a foundational operation in machine learning and scientific computing.<n>As KMVM tends to scale quadratically in both memory and time, applications are often limited by these computational constraints.<n>We propose a novel approximation procedure coined textitFaster-Fast and Free Memory Method ($fthreem$) to address these scaling issues.
  arXiv  Detail & Related papers  (2022-02-02T15:28:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.