Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control

• URL: http://arxiv.org/abs/2608.12123v1
• Date: Wed, 12 Aug 2026 14:42:15 GMT
• Title: Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
• Abstract summary: We formalize the ready-cohort boundary using fixed-share F, exact offline share P*, local upper bound U, and online achieved share A.<n>A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.

Related papers

• Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning [67.04448659688579]
  FGLGuard cuts AgentDojo's ground-success rate by 43% at near-unguarded utility, zero API cost, and negligible capability loss.<n>Federation is not optional: off-the-shelf transfer collapses under distribution shift.<n>Live FGLGuard cuts AgentDojo's ground-success rate by 43% at near-unguarded utility, zero API cost, and negligible capability loss.
  arXiv  Detail & Related papers  (2026-09-02T07:57:12Z)
• Metis: Typed Runtime Mediation for Tool-Using Software Agents [13.543030296451661]
  We present Metis, a multi-provider runtime that converts provider streams into typed events before admitted calls reach external effects.<n>We evaluate these mechanisms on frozen source artifacts.
  arXiv  Detail & Related papers  (2026-08-26T03:10:03Z)
• LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training [28.882595607766916]
  Training large language models on limited hardware is increasingly a scheduling problem.<n>We propose LazyTrain, an optimization layer over a layer-streaming executor.
  arXiv  Detail & Related papers  (2026-08-12T11:04:06Z)
• PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud [19.027069678233655]
  PhyAI is a Physical AI inference engine with a single runtime.<n>It runs vision-language-action (VLA) models and world-action models (WAMs) on single or multiple GPU.<n>It achieves 1.40x-4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot.
  arXiv  Detail & Related papers  (2026-08-04T13:53:48Z)
• HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference [47.0879800334801]
  HetRoute is a heterogeneous-cost-aware collaborative routing framework for distributed edge MoE inference.<n>HetRoute reduces average inference latency by up to 59.0% and P99 latency by up to 58.0%.
  arXiv  Detail & Related papers  (2026-08-01T10:30:01Z)
• CUADebug: Diagnosing and Repairing Computer-Use Agent Failures [65.56066380320107]
  Computer-use agents (CUAs) operate real desktop and web interfaces through screenshots, mouse and keyboard actions, and stateful UI feedback.<n>Unlike text-only agents, CUA failures arise from coupled visual perception, grounding, low-level interaction, task reasoning, and environment dynamics.<n>We introduce CUA Debug, a framework for diagnosing and repairing CUA failures.
  arXiv  Detail & Related papers  (2026-07-31T13:59:45Z)
• Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search [44.70501115891534]
  We present an automated mobile deployment pipeline that closes the loop from QLoRA fine-tuning of an architecture-generating LLM.<n>We evaluate the same frozen protocol on two benchmarks, CIFAR-10 and CIFAR-100, on a Samsung SM-P613 tablet.
  arXiv  Detail & Related papers  (2026-07-29T14:36:45Z)
• FusionML: Prefill, Not Decode - Mechanism and Boundaries of CPU+GPU Co-Execution on Unified-Memory Apple Silicon [0.0]
  Previous attempts, including our own, failed or produced precision-confounded wins.<n>We identify the cause: MLX's lazy-graph scheduler emphserializes cross-stream workWhenever a CPU-stream operation consumes an unmaterialized GPU result inside one evaluation graph, a row-split matmul runs x1.38 faster with materialized inputs.<n>We characterize the boundaries equally carefully: decode cannot benefit, bound by shared bandwidth co-execution does not add.
  arXiv  Detail & Related papers  (2026-07-24T11:42:17Z)
• DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners? [57.585275546688116]
  Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents.<n>We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.<n>We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt.
  arXiv  Detail & Related papers  (2026-06-10T17:58:49Z)
• MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research [39.68394140712043]
  MobileGym is a lightweight, fully controllable environment for everyday mobile use.<n>A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale.<n>A single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense rewards.
  arXiv  Detail & Related papers  (2026-05-25T17:59:49Z)
• Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment [60.442064966340524]
  We present a prototype study on NVIDIA Jetson Orin Nano devices connected over WiFi.<n>Key finding is that the dominant bottleneck is not just network bandwidth but also the CPU- GPU staging during communication.<n>Experiments show that this strategy reduces latency by 65%-77% and energy consumption by 34%-52% relative to full-tensor exchange.
  arXiv  Detail & Related papers  (2026-05-25T10:39:28Z)
• WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [88.10947115397971]
  This work presents WildClawBench, a native-runtime benchmark of 60 human-authored, bilingual, multimodal tasks spanning six thematic categories.<n>Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls.<n> Grading is hybrid, combining deterministic rule-based checks, environment-state auditing of side effects, and an LLM/VLM judge for semantic verification.
  arXiv  Detail & Related papers  (2026-05-11T17:49:43Z)
• TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition [0.0]
  This work presents TaxBreak, a trace-driven methodology for decomposing host-visible orchestration overhead.<n>We validate TaxBreak on NVIDIA H100 and H200 systems and use it to derive our proposed Host-Device Balance Index (HDBI)<n>We show that MoE models dispatch 8-11x more kernels per output token than dense models, and that for such host-bound workloads, single-thread performance is a first-order parameter.
  arXiv  Detail & Related papers  (2026-03-12T21:30:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.