FlowCompile: An Optimizing Compiler for Structured LLM Workflows
- URL: http://arxiv.org/abs/2605.13647v1
- Date: Wed, 13 May 2026 15:06:36 GMT
- Title: FlowCompile: An Optimizing Compiler for Structured LLM Workflows
- Abstract summary: FlowCompile is a structured LLM compiler that performs compile-time design space exploration to identify a high-quality, reusable trade-off set.<n> Experiments show that FlowCompile consistently outperforms optimized workflow configurations and routing-based baselines, delivering up to 6.4x speedup.
- Score: 44.34248788469801
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Structured LLM workflows, where specialized LLM sub-agents execute according to a predefined graph, have become a powerful abstraction for solving complex tasks. Optimizing such workflows, i.e., selecting configurations for each sub-agent to balance accuracy and latency, is challenging due to the combinatorial design space over model choices, reasoning budgets, and workflow structures. Existing cost-aware methods largely treat workflow optimization as a routing problem, selecting a configuration at inference time for each query according to the accuracy-latency objective used during training. We argue that structured LLM workflows can also be optimized from a compilation perspective: before deployment, the system can globally explore the workflow design space and construct a reusable set of workflow-level configurations spanning diverse accuracy-latency trade-offs. Drawing inspiration from machine learning compilers, we introduce FlowCompile, a structured LLM workflow compiler that performs compile-time design space exploration to identify a high-quality, reusable trade-off set. FlowCompile decomposes a workflow into sub-agents, profiles each sub-agent under diverse configurations, and composes these measurements through a structure-aware proxy to estimate workflow-level accuracy and latency. It then identifies diverse high-quality configurations in a single compile-time pass, without retraining or online adaptation. Experiments across diverse workflows and challenging benchmarks show that FlowCompile consistently outperforms heuristically optimized workflow configurations and routing-based baselines, delivering up to 6.4x speedup. The compiled configuration set further serves as a reusable optimization artifact, enabling flexible deployment under varying runtime preferences and supporting downstream selection or routing.
Related papers
- FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients [33.54212868461948]
Existing approaches for building powerful AI systems generally rely on human-crafted pipelines and prompts.<n>This paper describes a data-driven approach for automatically inducing agents and workflow.<n>We find that LLM discovered through our textscFlowBot induction performs competitively against strong baselines.
arXiv Detail & Related papers (2026-04-29T03:37:56Z) - From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents [39.457720579458]
Large language model (LLM)-based systems are becoming increasingly popular for solving tasks.<n>This survey reviews recent methods for designing and optimizing such computation.
arXiv Detail & Related papers (2026-03-23T17:42:59Z) - Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs [62.17306142810532]
We present Chimera, a predictive scheduling system for multi-agent workflow serving on heterogeneous LLM clusters.<n>Chime traces the best latency, reducing end-to-end latency by 1.2--2.4$times$ and improving task performance by 8.0-9.5 percentage points on average over competitive baselines.
arXiv Detail & Related papers (2026-03-23T17:01:42Z) - FlowMind: Execute-Summarize for Structured Workflow Generation from LLM Reasoning [5.153212048436295]
LLMs can solve complex tasks through reasoning and tool use, but accurately translating these solutions into structured remains challenging.<n>We model as sequences of tool use and reformulate the problem as designing a mechanism that can both solve tasks and reliably construct them.<n>We propose an Execute-Summarize(ES) framework that decouples task execution from workflow construction.
arXiv Detail & Related papers (2026-02-12T10:04:42Z) - Learning to Compose for Cross-domain Agentic Workflow Generation [56.630382886594184]
We create an open-source LLM for cross-domain workflow generation.<n>We learn a compact set of reusable workflow capabilities across diverse domains.<n>Our 1-pass generator surpasses SOTA refinement baselines that consume 20 iterations.
arXiv Detail & Related papers (2026-02-11T18:27:22Z) - Understanding and Optimizing Agentic Workflows via Shapley value [49.508008396810624]
We introduce ShapleyFlow, the first framework that employs cooperative game theory to analyze and optimize agentic configurations.<n>ShagleyFlow enables fine-grained attribution of each component's contribution and facilitates the identification of task-specific optimal configurations.
arXiv Detail & Related papers (2025-02-01T18:07:34Z) - Flow: Modularized Agentic Workflow Automation [53.073598156915615]
Multi-agent frameworks powered by large language models (LLMs) have demonstrated great success in automated planning and task execution.<n>However, the effective adjustment of agentic during execution has not been well studied.<n>In this paper, we define an activity-on-vertex (AOV) graph, which allows continuous workflow refinement by agents.<n>Our proposed multi-agent framework achieves efficient concurrent execution of subtasks, effective goal achievement, and enhanced error tolerance.
arXiv Detail & Related papers (2025-01-14T04:35:37Z) - Benchmarking Agentic Workflow Generation [80.74757493266057]
We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
arXiv Detail & Related papers (2024-10-10T12:41:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.