RuleFlow : Generating Reusable Program Optimizations with LLMs

• URL: http://arxiv.org/abs/2602.09051v1
• Date: Fri, 06 Feb 2026 21:08:33 GMT
• Title: RuleFlow : Generating Reusable Program Optimizations with LLMs
• Abstract summary: We introduce a hybrid approach that works in a 3-stage manner that decouples discovery from deployment.<n>We demonstrate that RuleFlow is the new state-of-the-art (SOTA) Pandas optimization framework on PandasBench.
• Score: 2.862995297420668
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optimizing Pandas programs is a challenging problem. Existing systems and compiler-based approaches offer reliability but are either heavyweight or support only a limited set of optimizations. Conversely, using LLMs in a per-program optimization methodology can synthesize nontrivial optimizations, but is unreliable, expensive, and offers a low yield. In this work, we introduce a hybrid approach that works in a 3-stage manner that decouples discovery from deployment and connects them via a novel bridge. First, it discovers per-program optimizations (discovery). Second, they are converted into generalised rewrite rules (bridge). Finally, these rules are incorporated into a compiler that can automatically apply them wherever applicable, eliminating repeated reliance on LLMs (deployment). We demonstrate that RuleFlow is the new state-of-the-art (SOTA) Pandas optimization framework on PandasBench, a challenging Pandas benchmark consisting of Python notebooks. Across these notebooks, we achieve a speedup of up to 4.3x over Dias, the previous compiler-based SOTA, and 1914.9x over Modin, the previous systems-based SOTA. Our code is available at https://github.com/ADAPT-uiuc/RuleFlow.

Related papers

• GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra [13.9407640377773]
  This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA)<n>Rlaopt addresses this gap by providing GPU-enabled solvers for positive-definite linear systems and convex empirical risk minimization with constraints and regularizers.<n>Rlaopt automatically checks compatibility with the selected solver and performs the required problem decomposition.
  arXiv  Detail & Related papers  (2026-09-08T02:14:05Z)
• Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation [2.2917707112773593]
  Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet quality standards.<n>It is unclear whether LLM code alignment could support both functional requirements (producing executable, correct code) and non-functional requirements (code readability, style, maintainability)<n>It is also unknown whether alignment for a code LLM should begin with base pretrained version or the finetuned (i.e., instruction-tuned) version of the LLM.
  arXiv  Detail & Related papers  (2026-06-27T16:22:22Z)
• Formalize, Don't Optimize: The Heuristic Trap in LLM-Generated Combinatorial Solvers [52.23061619664667]
  Large Language Models (LLMs) struggle to solve complex problems through direct reasoning, so recent neuro-symbolic systems increasingly use them to synthesize executable solvers.<n>We introduce CP-SynC-XL, a benchmark of 100 problems (4,577 instances), and evaluate three solver--construction paradigms: native algorithmic search (Python), constraint modeling through a Python solver API (Python + OR-Tools), and declarative constraint modeling.
  arXiv  Detail & Related papers  (2026-05-12T17:15:45Z)
• An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scale [17.725823339805697]
  This work revisits and improves on the ideas behind Landing, enabling the inclusion of modern adaptives.<n>Remarkably, these improvements come at little to no cost, and reduce the number of required hyperparemeters.<n>Our algorithm POGO is fast and GPU-friendly, consisting of only 5 matrix products.
  arXiv  Detail & Related papers  (2026-02-16T11:27:04Z)
• Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization [54.723834588133165]
  We study online linear optimization with matrix variables by the operatorAML, a setting where the geometry renders designing datadependent and efficient adaptive algorithms challenging.<n>We instantiate this framework with two efficient methods that avoid projections.<n>We show both methods admit closed-form updates match one-sided Shampoo's regret up to a constant factor, while significantly reducing computational cost.
  arXiv  Detail & Related papers  (2026-02-09T03:09:47Z)
• Panther: Faster and Cheaper Computations with Randomized Numerical Linear Algebra [0.0]
  We present Panther, a PyTorch-compatible library that consolidates established RandNLA algorithms into a single high-performance framework.<n>We demonstrate the effectiveness of RandNLA techniques and Panther's ease of adoption.
  arXiv  Detail & Related papers  (2026-01-21T21:23:00Z)
• StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold [51.93627542334909]
  Low-rank adaptation (LoRA) has been widely adopted as a parameter-efficient technique for fine-tuning large-scale pre-trained models.<n>We propose a geometry-aware extension of LoRA that uses a three-factor decomposition $U!SVtop$.
  arXiv  Detail & Related papers  (2025-10-02T11:59:13Z)
• Loop Invariant Generation: A Hybrid Framework of Reasoning optimised LLMs and SMT Solvers [2.0686733932673604]
  We investigate whether modern, reasoning-optimized large language models can do better than symbolic approaches.<n>We integrate OpenAI's O1, O1-mini, and O3-mini into a tightly coupled generate-and-check pipeline with the Z3 SMT solver.<n>Our framework achieves 100% coverage (133 out of 133), outperforming the previous best of 107 out of 133.
  arXiv  Detail & Related papers  (2025-08-01T08:15:15Z)
• Rewriting Pre-Training Data Boosts LLM Performance in Math and Code [21.587613905318705]
  We introduce two datasets, released under the Llama 3.3 Community License, that significantly enhance large language models (LLMs) performance.<n> SwallowCode refines Python snippets through a novel four-stage pipeline: syntax validation, pylint-based style filtering, and a two-stage rewriting process.<n>We show that continual pre-training of Llama-3.1-8B with SwallowCode boosts pass@1 by +17.0 on HumanEval and +17.7 on HumanEval+ compared to Stack-Edu.
  arXiv  Detail & Related papers  (2025-05-05T07:38:43Z)
• Training Deep Learning Models with Norm-Constrained LMOs [56.00317694850397]
  We propose a new family of algorithms that uses the linear minimization oracle (LMO) to adapt to the geometry of the problem.<n>We demonstrate significant speedups on nanoGPT training using our algorithm, Scion, without any reliance on Adam.
  arXiv  Detail & Related papers  (2025-02-11T13:10:34Z)
• Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient [57.9629676017527]
  We propose an optimization-based structural pruning that learns the pruning masks in a probabilistic space directly by optimizing the loss of the pruned model.<n>We achieve this by learning an underlying Bernoulli distribution to sample binary pruning masks.<n>Experiments conducted on LLaMA, LLaMA-2, LLaMA-3, Vicuna, and Mistral models demonstrate the promising performance of our method in efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-06-15T09:31:03Z)
• Scaling Sparse Fine-Tuning to Large Language Models [67.59697720719672]
  Large Language Models (LLMs) are difficult to fully fine-tune due to their sheer number of parameters. We propose SpIEL, a novel sparse finetuning method which maintains an array of parameter indices and the deltas of these parameters relative to their pretrained values. We show that SpIEL is superior to popular parameter-efficient fine-tuning methods like LoRA in terms of performance and comparable in terms of run time.
  arXiv  Detail & Related papers  (2024-01-29T18:43:49Z)
• ACPO: AI-Enabled Compiler Framework [1.752593459729982]
  This paper presents ACPO: An AI-Enabled Compiler Framework.<n>It provides LLVM with simple and comprehensive tools to benefit from employing ML models for different optimization passes.<n>We show that ACPO can provide a combined speedup of 4.5% on Polybench and 2.4% on Cbench when compared with LLVM's O3.
  arXiv  Detail & Related papers  (2023-12-15T17:49:24Z)
• ELRA: Exponential learning rate adaption gradient descent optimization method [83.88591755871734]
  We present a novel, fast (exponential rate), ab initio (hyper-free) gradient based adaption. The main idea of the method is to adapt the $alpha by situational awareness. It can be applied to problems of any dimensions n and scales only linearly.
  arXiv  Detail & Related papers  (2023-09-12T14:36:13Z)
• LCE: An Augmented Combination of Bagging and Boosting in Python [45.65284933207566]
  lcensemble is a high-performing, scalable and user-friendly Python package for the general tasks of classification and regression. Local Cascade Ensemble (LCE) is a machine learning method that further enhances the prediction performance of the current state-of-the-art methods Random Forest and XGBoost.
  arXiv  Detail & Related papers  (2023-08-14T16:34:47Z)
• Projection Efficient Subgradient Method and Optimal Nonsmooth Frank-Wolfe Method [54.93433440034386]
  We find a feasible $epsilon$-suboptimal solution using only $O(epsilon-1)$ PO calls and optimal $O(epsilon-2)$ FO calls. Our experiments confirm that these methods achieve significant speedups over the state-of-the-art, for a problem with costly PO and LMO calls.
  arXiv  Detail & Related papers  (2020-10-05T08:16:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.