A Multi-Dimensional, Per-Pass Empirical Study of the LLVM Optimization Pipeline

• URL: http://arxiv.org/abs/2606.31238v1
• Date: Tue, 30 Jun 2026 07:10:48 GMT
• Title: A Multi-Dimensional, Per-Pass Empirical Study of the LLVM Optimization Pipeline
• Abstract summary: We present a systematic, empirical study of the LLVM -O3 optimization pipeline.<n>We measure execution time, compile time, binary size, hardware counters, and RAPL energy across 84,750 measurements.<n>These findings enable more informed pass pruning, cost-model calibration, and autotuning.
• Score: 5.6148382426295305
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Quantifying the marginal impact of individual optimization passes underpins phase ordering, pass selection, optimization design, and analysis of pass/hardware interactions. In LLVM -- the standard backend for C/C++, Rust, and ML stacks via MLIR -- interactions among optimization passes, measurement noise, and pipeline scale make this difficult. We present a systematic, empirical study of the LLVM -O3 optimization pipeline. We decompose the pipeline into cumulative per-pass prefixes. We then measure execution time, compile time, binary size, hardware counters, and RAPL energy across 84,750 measurements covering 113 cumulative prefixes of the -O3 pipeline evaluated on 30 PolyBench/C kernels under rigorous noise mitigation. On these compute-bound affine kernels, the pipeline is non-monotone (6.6-9.7% of transitions regress) and strongly back-loaded (the median non-regressing kernel needs 84.8% of the pipeline for 80% of its speedup). Most gains are driven by a small Pareto-dominant core of passes, while the final -O3 configuration is Pareto-dominated on (size, speedup) for 29 of 30 kernels. We further show that IR instruction count is an unreliable predictor of runtime, that runtime-targeted passes are de facto energy-targeted (30-60% savings), and that the search-free idealized-additive upper bound on losses due to phase interference is 46.35%. These findings enable more informed pass pruning, cost-model calibration, and autotuning.

Related papers

• LoopVAE: Recurrent Depth Across Scales for Visual Tokenization [3.805468327504537]
  LoopVAE reuses a scale- and loop-conditioned core within and across scales.<n>A four-block core executes 28 block applications per encoder or decoder.<n>A non-adversarial Transformer ablation with the same execution graph finds competitive PSNR and SSIM under global sharing.
  arXiv  Detail & Related papers  (2026-09-10T13:20:03Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• A Reproducible Software Workflow for Unanchored Approximate MUB Optimization: A Case Study in Dimension Six [5.114953719334754]
  The workflow is designed for portable execution across CPU, Apple MPS, and HPC backends.<n>We optimize unanchored configurations across 100 random seeds for basis counts n = 3, 4, 5, 6 in complex128 and complex64 arithmetic.<n>We embed the representative d = 6, n = 4 transition unitaries into three-qubit 8x8 unitaries and execute the resulting circuits on the 156-qubit Heron processor ibm-marrakesh.
  arXiv  Detail & Related papers  (2026-07-12T07:24:25Z)
• LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation [53.88286427894783]
  Vision Language Models (VLMs) have emerged in the robotic domain as a powerful tool that enables environmental perception with language context.<n>Yet, their computational footprint typically confines them to cloud execution, hindering low-latency inference with local deployment on resource-constrained robots.<n>We present a distillation strategy that transfers complex spatial-semantic reasoning from large frontier models into a lightweight, 4B- parameter local VLM for edge execution on embedded GPU devices.
  arXiv  Detail & Related papers  (2026-06-26T09:11:59Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• CART: Context-Anchored Recurrent Transformer -- A Parameter-Efficient Architecture with Learned Stability [0.0]
  CART (Context-Anchored Recurrent Transformer) is a parameter-efficient language model that reuses a single shared core block R times across depth.<n>We evaluate CART on single consumer GPUs in two stages: a 64-configuration screen at 3,000 steps, then 36 configurations (P=6, R in 6,8,10, three seeds) trained for 30,500 steps (1B tokens)<n>Two patterns hold across widths d in 256,512,768,1024: prelude depth P dominates loop count R, and the Stage-1 ranking of R reverses at full training (R=6 becomes best at
  arXiv  Detail & Related papers  (2026-05-31T23:26:27Z)
• ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning [58.54940026861599]
  textscChunkFT is a memory-efficient fine-tuning framework.<n>textscChunkFT enables gradient computation for arbitrary sub-tensors without modifying the network architecture.<n>textscChunkFT consistently outperforms existing memory-efficient baselines.
  arXiv  Detail & Related papers  (2026-05-20T13:44:44Z)
• PARM: Pipeline-Adapted Reward Model [60.769414637325326]
  Reward models (RMs) are central to aligning large language models (LLMs) with human preferences, powering advanced decoding strategies.<n>While most prior work focuses on single-step generation, real-world applications increasingly adopt multi-stage pipelines.<n>We investigate this through code generation for optimization, constructing a pipeline that integrates reward models into both formulation and solution stages.
  arXiv  Detail & Related papers  (2026-04-20T14:29:08Z)
• Forge-UGC: FX optimization and register-graph engine for universal graph compiler [2.587194279527956]
  Existing frameworks such as OpenVINO and ONNX often use opaque compilation pipelines.<n> Forge-UGC addresses this with a hardware-agnostic design that separates graph capture, optimization, intermediate representation lowering, and backend scheduling.
  arXiv  Detail & Related papers  (2026-04-14T04:39:16Z)
• Parameter Analysis and Optimization of Layer Fidelity for Quantum Processor Benchmarking at Scale [0.03499870393443267]
  Layer fidelity is a benchmark well-suited to assessing processor performance at scale.<n>We extend the analysis of the original layer fidelity manuscript to optimize parameters of the benchmark.
  arXiv  Detail & Related papers  (2025-10-19T16:18:26Z)
• DeepPrune: Parallel Scaling without Inter-trace Redundancy [53.62015294143274]
  Over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation.<n>We propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning.<n>Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient.
  arXiv  Detail & Related papers  (2025-10-09T17:24:54Z)
• Provably Optimal Quantum Circuits with Mixed-Integer Programming [0.0]
  We present a depth-aware optimization framework for quantum circuit compilation.<n>For exact synthesis of a target unitary, we formulate a mixed-integer linear program (MILP) that linearly global-phase equivalence.<n>To scale beyond exact MILP, we propose a novel rolling-Circuit optimization (RHO) that rolls primarily in time, caps the active-qubits, and enforces per-qubit closure.
  arXiv  Detail & Related papers  (2025-10-01T08:25:43Z)
• EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference [49.94169109038806]
  This paper introduces EPS-MoE, a novel expert pipeline scheduler for MoE that surpasses the existing parallelism schemes.<n>Our results demonstrate at most 52.4% improvement in prefill throughput compared to existing parallel inference methods.
  arXiv  Detail & Related papers  (2024-10-16T05:17:49Z)
• 8-bit Optimizers via Block-wise Quantization [57.25800395197516]
  Statefuls maintain statistics over time, e.g., the exponentially smoothed sum (SGD with momentum) or squared sum (Adam) of past values. This state can be used to accelerate optimization compared to plain gradient descent but uses memory that might otherwise be allocated to model parameters. In this paper, we develop first gradients that use 8-bit statistics while maintaining the performance levels of using 32-bit gradient states.
  arXiv  Detail & Related papers  (2021-10-06T15:43:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.