PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems

• URL: http://arxiv.org/abs/2608.22130v2
• Date: Mon, 31 Aug 2026 02:14:07 GMT
• Title: PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems
• Abstract summary: PropUQ-MAS is an error propagation-aware UQ framework for multi-agent systems.<n>It estimates each step's reliability by combining local uncertainty with uncertainty inherited from upstream messages.<n>PropUQ-MAS consistently improves UQ in MAS, with average relative gains of +6.10% in AUROC and+47.58% in PRR.
• Score: 8.348628811300268
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-based multi-agent systems (MAS) solve complex tasks through communication among role-specialized agents. However, inter-agent dependencies introduce reliability risks beyond isolated agent failures. For instance, errors in intermediate messages could be inherited and amplified by downstream agents. Existing uncertainty quantification (UQ) methods mainly target isolated responses or single-agent reasoning, and therefore fail to capture uncertainty propagation in MAS. To this end, we propose PropUQ-MAS, an error propagation-aware UQ framework that represents MAS execution as a communication-structured graph and estimates each step's reliability by combining local uncertainty with uncertainty inherited from upstream messages. Extensive experiments demonstrate that PropUQ-MAS consistently improves UQ in MAS, with average relative gains of +6.10% in AUROC and +47.58% in PRR.

Related papers

• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.69180947909148]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems [11.562923882714095]
  This paper introduces a semantic-uncertainty-guided orchestration approach, HASSUM.<n>It estimates uncertainty using semantic entropy and semantic density, which measure trust at the level of answer semantics rather than output probabilities.<n>Because the approach operates independently of any particular agent architecture, it can be integrated into a broad range of hierarchical and collaborative multi-agent systems.
  arXiv  Detail & Related papers  (2026-08-11T03:55:53Z)
• Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks [0.0]
  This paper investigates how multi-agent systems can support actuarial risk modelling.<n>The main contribution is a novel approach to uncertainty propagation using token-level log-probabilities and a Bayesian Network.
  arXiv  Detail & Related papers  (2026-07-28T15:39:43Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments [60.3427704389541]
  Large Language Models are being increasingly deployed as the decision-making core of autonomous agents.<n>Yet, in conversational benchmarks, these agents frequently fail due to the cascading effects of incorrect decision-making.<n>We present the Failure-Aware Meta-Agentic framework to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-28T02:21:53Z)
• Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition [10.210978141489003]
  Large Language Model-based Multi-Agent Systems (MAS) consistently outperform single-agent systems on complex tasks.<n>Existing Uncertainty Quantification methods fail to address the unique complexities of the MAS.<n>We introduce MATU, a novel framework that quantifies uncertainty through tensor decomposition.
  arXiv  Detail & Related papers  (2026-04-09T19:01:50Z)
• Towards Reducible Uncertainty Modeling for Reliable Large Language Model Agents [72.26774492844167]
  Uncertainty quantification (UQ) for large language models (LLMs) is a key building block for safety guardrails of daily LLM applications.<n>This paper presents the first general formulation of agent UQ that subsumes broad classes of existing UQ setups.<n>We propose a novel perspective, a conditional uncertainty reduction process, that explicitly models reducible uncertainty over an agent's trajectory.
  arXiv  Detail & Related papers  (2026-02-04T21:47:40Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• AgentAsk: Multi-Agent Systems Need to Ask [26.13279490836716]
  Multi-agent systems built on large language models (LLMs) promise enhanced problem-solving capabilities through collaborative division of labor.<n>We propose AgentAsk, a lightweight and plug-and-play clarification module that treats every inter-agent message as a potential failure point and inserts minimally necessary questions to arrest error propagation.<n>AgentAsk consistently improves accuracy and robustness over public multi-agent implementations while keeping overhead minimal, with latency and extra cost all less than 5%.
  arXiv  Detail & Related papers  (2025-10-08T22:36:05Z)
• Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency [66.96286531087549]
  Uncertainty quantification (UQ) methods for Large Language Models (LLMs) encompass a variety of approaches.<n>We propose a novel approach to integrating model confidence with output consistency, resulting in a family of efficient and robust UQ methods.<n>We evaluate our approach across various tasks such as question answering, abstractive summarization, and machine translation.
  arXiv  Detail & Related papers  (2025-02-07T14:30:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.