A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems

• URL: http://arxiv.org/abs/2608.05791v1
• Date: Thu, 06 Aug 2026 09:26:24 GMT
• Title: A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems
• Abstract summary: Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference.<n>This paper models parallelism in multi-agent systems as two distinct levels of decision processes: Replica Parallelism and Structural Parallelism.<n>We propose TIPEX, an execution framework that unifies these two levels of parallelism and coordinates their roles within the inference process.
• Score: 8.810136864681994
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference, and their execution strategies directly affect system accuracy, latency, and computational cost. Parallel execution provides a means to improve inference-time efficiency. From the perspective of inference-time execution, this paper models parallelism in multi-agent systems as two distinct levels of decision processes: Replica Parallelism, which explores multiple complete solution paths at the task level, and Structural Parallelism, which enables concurrent execution within a single solution path through task decomposition. However, the roles of different forms of parallelism and their interrelationships still lack systematic study in terms of unified organization and coordination. We therefore propose TIPEX, a controllable execution framework that unifies these two levels of parallelism and coordinates their roles within the inference process under a unified execution semantics while supporting systematic combinations and analyses of different parallel strategies and parameter configurations. Systematic experiments on the GAIA benchmark demonstrate that inference-time parallelism can significantly improve accuracy and reduce end-to-end latency at the cost of increased token consumption. Further analysis shows that Replica and Structural Parallelism exhibit complementary effects across task complexities, with tasks of intermediate difficulty benefiting most from their coordination, while overly aggressive parallel strategies do not necessarily yield better performance.

Related papers

• EASy: Towards Efficient LLM-Based Agentic System [54.10819421625103]
  We propose EASy, a trainable agentic framework that jointly optimize task performance and computational efficiency through reinforcement learning.<n> EASy equips an LLM-based orchestrator with explicit knowledge of the capability and cost profiles of heterogeneous executors, enabling context-sensitive coordination beyond performance-only routing.<n>It further introduces a milestone-plan-act workflow that decomposes complex tasks into manageable milestones, constructs dependency-aware execution graphs, assigns suitable executors, and parallelizes independent steps while adapting subsequent decisions to intermediate outcomes.
  arXiv  Detail & Related papers  (2026-08-05T08:50:36Z)
• HSAP: A Hierarchical Sequence-aware Parallelism for Hybrid-Context Generative Models [17.09673128667358]
  We propose an efficient Sequence-Aware Parallelism algorithm to conquer the obstacles of intensive tensor transmission and partial attention computation.<n>Our algorithm utilizes JIT compilation to optimize the communication strategy of all device groups in NCCL level.
  arXiv  Detail & Related papers  (2026-06-29T15:26:55Z)
• Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition [61.291733522717415]
  Agent-as-Tool is a unified parallel orchestration paradigm that mitigates both agents and tools into a standardized, learnable action space.<n>ParaManager decouples planning decisions from subtask solving, enabling state-aware parallel subtask decomposition, delegation, and asynchronous execution.<n>Experiments show that ParaManager achieves strong performance across multiple benchmarks and exhibits robust generalization under unseen model pools.
  arXiv  Detail & Related papers  (2026-04-18T14:41:27Z)
• Para-B&B: Load-Balanced Deterministic Parallelization of Solving MIP [50.917107318582715]
  Mixed-integer programming (MIP) extends linear programming by incorporating both continuous and integer decision variables.<n>This paper presents the first fully open-source implementation of deterministic parallel branch-and-bound for HiGHS, a high-performance MIP solver.<n>Our approach introduces a novel data-parallel architecture ensuring strict determinism by replicating complete solver state across worker threads.
  arXiv  Detail & Related papers  (2026-02-10T14:17:53Z)
• Learning to Share: Selective Memory for Efficient Parallel Agentic Systems [49.78267008828593]
  Agentic systems solve complex tasks by coordinating multiple agents that iteratively reason, invoke tools, and exchange intermediate results.<n>Recent approaches deploy multiple agent teams running in parallel to explore diverse reasoning trajectories.<n>We propose Learning to Share (LTS), a learned shared-memory mechanism for parallel agentic frameworks.
  arXiv  Detail & Related papers  (2026-02-05T18:20:21Z)
• Learning Latency-Aware Orchestration for Parallel Multi-Agent Systems [18.192867631682674]
  We investigate learning-based orchestration of multi-agent systems with explicit latency under parallel execution.<n>We propose latency-aware Multi-agent System (LAMaS), a work-aware multi-agent orchestration framework.<n>Our experiments show that our approach reduces critical path length by 38-46% compared to the state-of-the-art baseline for multi-agent architecture search.
  arXiv  Detail & Related papers  (2026-01-15T16:23:53Z)
• Modular Task Decomposition and Dynamic Collaboration in Multi-Agent Systems Driven by Large Language Models [3.4219049032524804]
  This paper addresses the limitations of a single agent in task decomposition and collaboration during complex task execution.<n>It proposes a multi-agent architecture for modular task decomposition and dynamic collaboration based on large language models.
  arXiv  Detail & Related papers  (2025-11-03T02:00:06Z)
• Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Models [99.85131798240808]
  We introduce a novel generative framework called textitGuided Topology Diffusion (GTD)<n>Inspired by conditional discrete graph diffusion models, GTD formulates topology synthesis as an iterative construction process.<n>At each step, the generation is steered by a lightweight proxy model that predicts multi-objective rewards.<n>Experiments show that GTD can generate highly task-adaptive, sparse, and efficient communication topologies.
  arXiv  Detail & Related papers  (2025-10-09T05:28:28Z)
• Optimizing Sequential Multi-Step Tasks with Parallel LLM Agents [15.26802977779826]
  M1-Parallel is a framework that concurrently runs multiple multi-agent teams in parallel to uncover distinct solution paths.<n>We show that M1-Parallel with early termination achieves up to $2.2times$ speedup while preserving accuracy.<n>We further investigate strategies aimed at encouraging diverse execution plans but observe no additional performance gains over repeated sampling.
  arXiv  Detail & Related papers  (2025-07-11T18:09:22Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• Parallelized Planning-Acting for Efficient LLM-based Multi-Agent Systems [31.894636711684523]
  We propose a novel parallelized planning-acting framework for Multi-Agent Systems.<n>The proposed framework features a dual-thread architecture with interruptible execution to enable concurrent planning and acting.
  arXiv  Detail & Related papers  (2025-03-05T13:53:10Z)
• A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems [54.61091936472494]
  We develop a two-stage framework, in which reinforcement learning (RL) and traditional operations research (OR) algorithms are combined together. The scheduling problem is solved in two stages, including a finite Markov decision process (MDP) and a mixed-integer programming process, respectively. Results show that the proposed algorithms could stably and efficiently obtain satisfactory scheduling schemes for agile Earth observation satellite scheduling problems.
  arXiv  Detail & Related papers  (2021-03-10T03:16:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.