Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows

• URL: http://arxiv.org/abs/2609.10964v1
• Date: Thu, 10 Sep 2026 01:35:10 GMT
• Title: Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows
• Abstract summary: tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next.<n>We present a tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next.
• Score: 25.368121854431035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic LLM workflows consist of sequences of model turns interleaved with tool interactions, so their end-to-end completion time depends not only on inference speed but also on when ready turns are released. Most runtimes release each turn immediately upon readiness. Under contention, this eager release policy can accumulate released but unfinished work; once submitted, those turns can no longer be reordered by the workflow-level policy, increasing tail latency. We present a tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next and how much released but unfinished work to maintain. The method uses a mean--Conditional Value-at-Risk (CVaR) objective to capture the evolving tail risk of unfinished workflows, incorporates online estimates of turn work when prioritizing ready turns, and adapts the released work budget to observed queue pressure. We evaluate the method using real agent execution traces from software engineering tasks across multiple LLMs and workflow arrival rates. The method performs comparably to eager release under light load and substantially reduces the P95 of workflow flow time under contention, achieving up to a \(3.50\times\) speedup.

Related papers

• Substrate-Portable Execution for Production LLM Workflows [0.0]
  Execution semantics are often coupled to one runtime.<n>Rufus supports real-time serving, asynchronous background tasks, and high-volume batch workloads.<n>We present a binding-adaptive agent execution platform that separates workflow definition from execution substrate.
  arXiv  Detail & Related papers  (2026-09-05T14:52:35Z)
• COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution [12.755574564434788]
  We propose COVENANT, a compiler-and-interpreter architecture for workflow-aligned agent execution.<n>Our key insight is to treat workflow instructions as source programs rather than prompts.<n>COVENANT converts the instructions into a workflow abstract syntax tree (WAST) and lowers it to a workflow control-flow graph (WCFG)
  arXiv  Detail & Related papers  (2026-07-28T07:59:50Z)
• On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows [37.22092976146466]
  Agentic systems increasingly solve complex user requests by executing orchestrated executables, where subtasks are assigned to specialized models or tools and coordinated according to their dependencies.<n>Real deployments often impose concrete requirements: a workflow must be completed within a specified budget and before a specified deadline.<n>This shifts the goal from average efficiency optimization to maximizing the probability that the entire workflow completes successfully under explicit budget and deadline constraints.
  arXiv  Detail & Related papers  (2026-05-07T12:24:21Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• VineLM: Trie-Based Fine-Grained Control for Agentic Workflows [4.984467399543813]
  VineLM represents feasible executions as an annotated trie of model-choice prefixes.<n>VineLM represents feasible executions as an annotated trie of model-choice prefixes.
  arXiv  Detail & Related papers  (2026-04-09T15:31:22Z)
• B-PASTE: Beam-Aware Pattern-Guided Speculative Execution for Resource-Constrained LLM Agents [0.0]
  LLM agents execute in an interleaved reasoning-and-action loop, where future tool calls cannot be launched until the current reasoning step completes.<n>We propose B-PASTE, a beam-aware extension that lifts speculation from single tools to local branch hypotheses under strict resource constraints.
  arXiv  Detail & Related papers  (2026-04-09T07:42:17Z)
• Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs [62.17306142810532]
  We present Chimera, a predictive scheduling system for multi-agent workflow serving on heterogeneous LLM clusters.<n>Chime traces the best latency, reducing end-to-end latency by 1.2--2.4$times$ and improving task performance by 8.0-9.5 percentage points on average over competitive baselines.
  arXiv  Detail & Related papers  (2026-03-23T17:01:42Z)
• FlowSteer: Interactive Agentic Workflow Orchestration via End-to-End Reinforcement Learning [49.369614288007334]
  FlowSteer is an end-to-end reinforcement learning framework that takes a lightweight policy model as the agent and an executable canvas environment.<n>We show that FlowSteer significantly outperforms baselines across various tasks.
  arXiv  Detail & Related papers  (2026-02-02T05:30:42Z)
• Flow: Modularized Agentic Workflow Automation [53.073598156915615]
  Multi-agent frameworks powered by large language models (LLMs) have demonstrated great success in automated planning and task execution.<n>However, the effective adjustment of agentic during execution has not been well studied.<n>In this paper, we define an activity-on-vertex (AOV) graph, which allows continuous workflow refinement by agents.<n>Our proposed multi-agent framework achieves efficient concurrent execution of subtasks, effective goal achievement, and enhanced error tolerance.
  arXiv  Detail & Related papers  (2025-01-14T04:35:37Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)
• Better than the Best: Gradient-based Improper Reinforcement Learning for Network Scheduling [60.48359567964899]
  We consider the problem of scheduling in constrained queueing networks with a view to minimizing packet delay. We use a policy gradient based reinforcement learning algorithm that produces a scheduler that performs better than the available atomic policies.
  arXiv  Detail & Related papers  (2021-05-01T10:18:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.