MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems

• URL: http://arxiv.org/abs/2604.26963v1
• Date: Tue, 14 Apr 2026 05:15:28 GMT
• Title: MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
• Abstract summary: Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents.<n> coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge.<n>We design and implement MARS, an efficient and adaptive co-scheduling system.<n>Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput.
• Score: 22.470203759341004
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators. Agentic workloads induce a temporal shift from single-turn inference to multi-turn LLM-tool loops, and a spatial shift from chat-scale, GPU-only execution to repository-scale, GPU-CPU co-located execution. Consequently, coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge. We design and implement MARS, an efficient and adaptive co-scheduling system that globally coordinates heterogeneous agentic workloads under coupled GPU-CPU resource pressure. By establishing holistic visibility across GPU inference and CPU tool execution via a unified information stream, an external control plane in MARS decouples admission from execution to prevent heterogeneous resource oversubscription. An internal agent-centric scheduler further minimizes the end-to-end critical path by prioritizing latency-sensitive continuations and adaptively retaining KV cache state only when warm resumption yields a latency benefit. Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput. We further integrate MARS as the serving backend for the OpenHands coding agent framework, demonstrating its real-world effectiveness by accelerating end-to-end task completion time by up to 1.87x. Our source code will be publicly available soon.

Related papers

• End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services [13.04369901641143]
  Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference.<n>We develop a cross-slot inference model that captures transmission, prefill, redistribution-level decoding, and key-value cache evolution for each diverse request.<n>We propose the LYREO approach that transforms the long-term load-balancing constraint via Lyapunov optimization and employs reward with sequencebased return prediction.
  arXiv  Detail & Related papers  (2026-09-15T13:50:54Z)
• A-MADiff: Attention-Guided Multi-Agent DRL with Diffusion Policies for Memory-Aware Task Orchestration in Mobile AIGC Networks [12.516277671699525]
  Mobile AIGC networks host GenAI models on edge-located AIGC Service Providers (ASPs) to deliver low-latency and personalized AIGC services for mobile users.<n>Existing studies on AIGC task orchestration have largely overlooked GPU memory feasibility constraints.<n>We propose an underlineAttention-guided underlineMulti-underlineAgent deep reinforcement learning algorithm with underlineDiffusion policies (A-MADiff)<n> Numerical results demonstrate that A-MADiff significantly improves the cumulative reward over the state-of-the
  arXiv  Detail & Related papers  (2026-08-29T13:19:44Z)
• Multi-Agent Computer Use [72.79887808312706]
  We argue that we should move towards evaluating and building multi-agent computer use (MACU) systems.<n>We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG)<n>At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG.
  arXiv  Detail & Related papers  (2026-06-01T01:29:36Z)
• SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters [13.891522069967507]
  We propose a shift to program-level scheduling for compound AI workloads.<n>We present SAGA, a distributed scheduler that implements this abstraction through three mechanisms.
  arXiv  Detail & Related papers  (2026-05-01T09:05:28Z)
• TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition [0.0]
  This work presents TaxBreak, a trace-driven methodology for decomposing host-visible orchestration overhead.<n>We validate TaxBreak on NVIDIA H100 and H200 systems and use it to derive our proposed Host-Device Balance Index (HDBI)<n>We show that MoE models dispatch 8-11x more kernels per output token than dense models, and that for such host-bound workloads, single-thread performance is a first-order parameter.
  arXiv  Detail & Related papers  (2026-03-12T21:30:07Z)
• Towards Efficient Agents: A Co-Design of Inference Architecture and System [66.59916327634639]
  This paper presents AgentInfer, a unified framework for end-to-end agent acceleration.<n>We decompose the problem into four synergistic components: AgentCollab, AgentSched, AgentSAM, and AgentCompress.<n>Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%.
  arXiv  Detail & Related papers  (2025-12-20T12:06:13Z)
• Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap [15.352230356342366]
  Mixture-of-Agents (MoA) inference can suffer from dense inter-agent communication and low hardware utilization.<n>We present a serving design that targets these bottlenecks through an algorithm-system co-design.
  arXiv  Detail & Related papers  (2025-12-19T23:06:58Z)
• Astraea: A State-Aware Scheduling Engine for LLM-Powered Agents [12.884297990127985]
  Astraea is a service engine designed to shift the optimization from local segments to the global request lifecycle.<n>It employs a state-aware, hierarchical scheduling algorithm that integrates a request's historical state with future predictions.<n>Astraea reduces average JCT by up to 25.5% compared to baseline methods.
  arXiv  Detail & Related papers  (2025-12-16T06:55:10Z)
• Edge Collaborative Gaussian Splatting with Integrated Rendering and Communication [69.23838350582764]
  We present edge collaborative (ECO-GS) where each user can switch between a small GS model to guarantee fidelity and a remote large GS model to guarantee fidelity.<n>We propose integrated and communication (IRAC) which jointly optimize low-cost rendering status and edge power allocation.
  arXiv  Detail & Related papers  (2025-10-26T15:33:29Z)
• xLLM Technical Report [57.13120905321185]
  We introduce xLLM, an intelligent and efficient Large Language Model (LLM) inference framework.<n>xLLM builds a novel decoupled service-engine architecture.<n>xLLM-Engine co-optimizes system and algorithm designs to fully saturate computing resources.
  arXiv  Detail & Related papers  (2025-10-16T13:53:47Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC [11.82567747365518]
  This paper introduces Agent.xpu, an efficient serving system for agentic LLM workloads on memory-unified heterogeneous SOCs.<n>With dedicated offline profiling, Agent.xpu first constructs a heterogeneous execution graph, which fuses and chunks model kernels for affinity-guided, elastic accelerator mapping.<n>At runtime, its online scheduler enables fine-grained, kernel-level preemption to guarantee the responsiveness of reactive tasks.
  arXiv  Detail & Related papers  (2025-06-30T16:50:48Z)
• HPRM: High-Performance Robotic Middleware for Intelligent Autonomous Systems [1.7838625117601146]
  We present High-Performance Robotic Middleware (HPRM), built on top of the deterministic coordination language Lingua Franca (LF)<n>HPRM employs optimizations including an in-memory object store for efficient zero-copy transfer of large payloads, adaptive serialization to minimize serialization overhead, and an eager protocol with real-time sockets to reduce handshake latency.<n>We demonstrate the benefits of HPRM by integrating it with the CARLA simulator and running reinforcement learning agents along with object detection workloads.
  arXiv  Detail & Related papers  (2024-12-02T18:46:29Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.