Learning Latency-Aware Orchestration for Parallel Multi-Agent Systems

• URL: http://arxiv.org/abs/2601.10560v1
• Date: Thu, 15 Jan 2026 16:23:53 GMT
• Title: Learning Latency-Aware Orchestration for Parallel Multi-Agent Systems
• Abstract summary: We investigate learning-based orchestration of multi-agent systems with explicit latency under parallel execution.<n>We propose latency-aware Multi-agent System (LAMaS), a work-aware multi-agent orchestration framework.<n>Our experiments show that our approach reduces critical path length by 38-46% compared to the state-of-the-art baseline for multi-agent architecture search.
• Score: 18.192867631682674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent systems (MAS) enable complex reasoning by coordinating multiple agents, but often incur high inference latency due to multi-step execution and repeated model invocations, severely limiting their scalability and usability in time-sensitive scenarios. Most existing approaches primarily optimize task performance and inference cost, and explicitly or implicitly assume sequential execution, making them less optimal for controlling latency under parallel execution. In this work, we investigate learning-based orchestration of multi-agent systems with explicit latency supervision under parallel execution. We propose Latency-Aware Multi-agent System (LAMaS), a latency-aware multi-agent orchestration framework that enables parallel execution and explicitly optimizes the critical execution path, allowing the controller to construct execution topology graphs with lower latency under parallel execution. Our experiments show that our approach reduces critical path length by 38-46% compared to the state-of-the-art baseline for multi-agent architecture search across multiple benchmarks, while maintaining or even improving task performance. These results highlight the importance of explicitly optimizing latency under parallel execution when designing efficient multi-agent systems. The code is available at https://github.com/xishi404/LAMaS

Related papers

• Para-Pipe: Exploiting Hierarchical Operator Parallelism of ML Computational Graphs on SoCs [22.6523068033902]
  Para-Pipe is a hierarchical mapping framework that integrates intra- and inter-stage operator parallelism within a pipelined architecture.<n>Para-Pipe navigates the trade-off between throughput and latency by selectively fine-tuning parallelism levels within and across pipeline stages.
  arXiv  Detail & Related papers  (2026-09-03T17:53:44Z)
• PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents [79.28372522980153]
  PeakBench is a benchmark of executable multi-tool dependency annotations and measured resource profiles.<n>We show that strong logical planning does not reliably translate into safe or efficient execution under resource constraints.<n>PeakBench is a useful testbed for diagnosing resource-aware agent behavior.
  arXiv  Detail & Related papers  (2026-08-25T12:56:08Z)
• A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems [8.810136864681994]
  Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference.<n>This paper models parallelism in multi-agent systems as two distinct levels of decision processes: Replica Parallelism and Structural Parallelism.<n>We propose TIPEX, an execution framework that unifies these two levels of parallelism and coordinates their roles within the inference process.
  arXiv  Detail & Related papers  (2026-08-06T09:26:24Z)
• Multi-Agent Computer Use [72.79887808312706]
  We argue that we should move towards evaluating and building multi-agent computer use (MACU) systems.<n>We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG)<n>At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG.
  arXiv  Detail & Related papers  (2026-06-01T01:29:36Z)
• Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition [61.291733522717415]
  Agent-as-Tool is a unified parallel orchestration paradigm that mitigates both agents and tools into a standardized, learnable action space.<n>ParaManager decouples planning decisions from subtask solving, enabling state-aware parallel subtask decomposition, delegation, and asynchronous execution.<n>Experiments show that ParaManager achieves strong performance across multiple benchmarks and exhibits robust generalization under unseen model pools.
  arXiv  Detail & Related papers  (2026-04-18T14:41:27Z)
• InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seeking [39.22757710468795]
  We present framework, a hierarchical framework based on principle of near-decomposability, containing a strategic textitHost, multiple textitManagers and parallel textitWorkers.<n>By leveraging aggregation and reflection mechanisms at the Manager layer, our framework enforces strict context isolation to prevent saturation and error propagation.<n>Our evaluation on two complementary benchmarks demonstrates both efficiency ($ 3-5 times$ speed-up) and effectiveness, achieving a $8.4%$ success rate on WideSearch-en and $52.9%$ accuracy on BrowseComp-zh
  arXiv  Detail & Related papers  (2026-04-03T11:19:17Z)
• Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs [62.17306142810532]
  We present Chimera, a predictive scheduling system for multi-agent workflow serving on heterogeneous LLM clusters.<n>Chime traces the best latency, reducing end-to-end latency by 1.2--2.4$times$ and improving task performance by 8.0-9.5 percentage points on average over competitive baselines.
  arXiv  Detail & Related papers  (2026-03-23T17:01:42Z)
• Learning to Share: Selective Memory for Efficient Parallel Agentic Systems [49.78267008828593]
  Agentic systems solve complex tasks by coordinating multiple agents that iteratively reason, invoke tools, and exchange intermediate results.<n>Recent approaches deploy multiple agent teams running in parallel to explore diverse reasoning trajectories.<n>We propose Learning to Share (LTS), a learned shared-memory mechanism for parallel agentic frameworks.
  arXiv  Detail & Related papers  (2026-02-05T18:20:21Z)
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models [99.6720868215076]
  We introduce ThreadWeaver, a framework for adaptive parallel reasoning.<n> ThreadWeaver achieves accuracy on par with popular sequential reasoning models of comparable size.<n>We show that ThreadWeaver delivers up to 1.53x average speedup in token latency.
  arXiv  Detail & Related papers  (2025-11-24T18:55:59Z)
• FlashResearch: Real-time Agent Orchestration for Efficient Deep Research [62.03819662340356]
  FlashResearch is a novel framework for efficient deep research.<n>It transforms sequential processing into parallel, runtime orchestration.<n>It can deliver up to a 5x speedup while maintaining comparable quality.
  arXiv  Detail & Related papers  (2025-10-02T00:15:39Z)
• Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution [48.7788770680643]
  Flash-Searcher is a novel parallel agent reasoning framework.<n>It decomposes complex tasks into subtasks with explicit dependencies, enabling concurrent execution of independent reasoning paths.<n>It achieves 67.7% accuracy on BrowseComp and 83% on xbench-DeepSearch, while reducing agent execution steps by up to 35% compared to current frameworks.
  arXiv  Detail & Related papers  (2025-09-29T17:39:30Z)
• Accelerating Latency-Critical Applications with AI-Powered Semi-Automatic Fine-Grained Parallelization on SMT Processors [0.0]
  Simultaneous Multithreading (SMT) technology is rarely used with heavy threads of latency-critical applications.<n>We introduce Aira, an AI-powered Parallelization Adviser.<n>We show 17% geomean performance gain from parallelization of latency-critical benchmarks using Aira with Relic framework.
  arXiv  Detail & Related papers  (2025-08-31T14:51:19Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• Optimizing Sequential Multi-Step Tasks with Parallel LLM Agents [15.26802977779826]
  M1-Parallel is a framework that concurrently runs multiple multi-agent teams in parallel to uncover distinct solution paths.<n>We show that M1-Parallel with early termination achieves up to $2.2times$ speedup while preserving accuracy.<n>We further investigate strategies aimed at encouraging diverse execution plans but observe no additional performance gains over repeated sampling.
  arXiv  Detail & Related papers  (2025-07-11T18:09:22Z)
• Faster and Better LLMs via Latency-Aware Test-Time Scaling [47.3923926808606]
  Test-Time Scaling (TTS) has proven effective in improving the performance of Large Language Models (LLMs) during inference.<n>Existing research has overlooked the efficiency of TTS from a latency-sensitive perspective.<n>We demonstrate that a compute-optimal TTS does not always result in the lowest latency in scenarios where latency is critical.
  arXiv  Detail & Related papers  (2025-05-26T07:51:30Z)
• Multi-Bin Batching for Increasing LLM Inference Throughput [19.652542432683234]
  Large language models (LL) grow in popularity improving the efficiency of their systems.<n> requests is a critical step in scheduling jobs on servers.<n> requests often have varying generation lengths, causing resource underutilization.<n>We formalize this problem from a queueing-theoretic perspective, and aim to design a throughput control policy.
  arXiv  Detail & Related papers  (2024-12-03T03:16:12Z)
• PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation [9.080650575731152]
  PipeInfer is a pipelined speculative acceleration technique to reduce inter-token latency and improve system utilization for single-request scenarios. PipeInfer exhibits up to a 2.15$times$ improvement in generation speed over standard speculative inference.
  arXiv  Detail & Related papers  (2024-07-16T14:52:02Z)
• Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints [52.58352707495122]
  We present a multi-robot allocation algorithm that decouples the key computational challenges of sequential decision-making under uncertainty and multi-agent coordination. We validate our results over a wide range of simulations on two distinct domains: multi-arm conveyor belt pick-and-place and multi-drone delivery dispatch in a city.
  arXiv  Detail & Related papers  (2020-05-27T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.