Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems

• URL: http://arxiv.org/abs/2608.14707v1
• Date: Tue, 11 Aug 2026 03:55:53 GMT
• Title: Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems
• Abstract summary: This paper introduces a semantic-uncertainty-guided orchestration approach, HASSUM.<n>It estimates uncertainty using semantic entropy and semantic density, which measure trust at the level of answer semantics rather than output probabilities.<n>Because the approach operates independently of any particular agent architecture, it can be integrated into a broad range of hierarchical and collaborative multi-agent systems.
• Score: 11.562923882714095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language model (LLM)-based multi-agent systems become increasingly capable, coordinating agents under uncertainty becomes a fundamental challenge. Existing orchestration strategies typically rely on fixed interaction patterns and often lack mechanisms for assessing the reliability of intermediate reasoning steps, allowing errors and hallucinations to propagate through the system. This paper introduces a semantic-uncertainty-guided orchestration approach, HASSUM as a general framework for uncertainty-aware coordination in multi-agent systems. The method estimates uncertainty using semantic entropy and semantic density, which measure trust at the level of answer semantics rather than output probabilities. These signals enable adaptive orchestration decisions, including output verification, selective reprompting, additional deliberation, and confidence-aware response selection. Because the approach operates independently of any particular agent architecture, it can be integrated into a broad range of hierarchical and collaborative multi-agent systems. The evaluations demonstrate an implementation within a hierarchical agent framework and evaluate it on StrategyQA, JailbreakBench, and TruthfulQA benchmarks. Across tasks that require complex reasoning and are prone to ambiguity or hallucinations, uncertainty-guided orchestration yields more reliable outcomes than uncertainty-unaware coordination. Semantic entropy and semantic density in tandem outperformed either metric alone. Ablations testing different thresholds and model sizes demonstrated that both influence the effectiveness of semantic metrics. The results suggest that semantic uncertainty is a practical and general-purpose signal for improving robustness and trustworthiness in agentic AI systems.

Related papers

• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.44420701722915]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning [63.24624171679711]
  Large language model (LLM)-based agents often make suboptimal tool-use decisions.<n>We propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation.
  arXiv  Detail & Related papers  (2026-06-05T07:08:34Z)
• Learning to Choose: An Empowerment-Guided Multi-Agent System with semantic communication for Adaptive Method Selection [1.321203201549798]
  In multi-agent pipelines, small inconsistencies between agent intentions and actions can lead to semantic drift.<n>We introduce a multi-agent framework that combines contextual bandits with structured inter-agent communication and semantic checkpoints.<n>We show that reliable autonomous learning requires not only identifying high-quality actions, but also preserving the integrity of their propagation.
  arXiv  Detail & Related papers  (2026-05-28T14:57:21Z)
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security [57.35851886874902]
  Agentic AI systems execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness.<n>This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments.<n>For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies.
  arXiv  Detail & Related papers  (2026-05-17T10:26:37Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition [62.16431420189863]
  LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states.<n>We introduce AffectAgent, an affect-oriented multi-agent retrieval-augmented generation framework.<n>AffectAgent comprises three jointly optimized agents, namely a query planner, an evidence filter, and an emotion generator.
  arXiv  Detail & Related papers  (2026-04-14T13:49:19Z)
• Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition [10.210978141489003]
  Large Language Model-based Multi-Agent Systems (MAS) consistently outperform single-agent systems on complex tasks.<n>Existing Uncertainty Quantification methods fail to address the unique complexities of the MAS.<n>We introduce MATU, a novel framework that quantifies uncertainty through tensor decomposition.
  arXiv  Detail & Related papers  (2026-04-09T19:01:50Z)
• COMPASS: The explainable agentic framework for Sovereignty, Sustainability, Compliance, and Ethics [0.0]
  This paper introduces a novel multi-agent orchestration system designed to enforce value-aligned AI.<n>The framework comprises an Orchestrator and four specialised sub-agents addressing sovereignty, carbon-aware computing, compliance, and ethics.<n>By employing an LLM-as-a-judge methodology, the system assigns quantitative scores and generates explainable justifications for each assessment dimension.
  arXiv  Detail & Related papers  (2026-03-11T20:14:35Z)
• The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI [0.0]
  This paper introduces a novel auditing framework to quantify latent trait estimation under ordinal uncertainty.<n>The research audits nine leading models across dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization.
  arXiv  Detail & Related papers  (2026-02-19T06:56:01Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution [0.7226144684379191]
  We present ECHO, a novel algorithm that combines hierarchical context representation, objective analysis-based evaluation, and consensus voting to improve error attribution accuracy.<n> Experimental results demonstrate that ECHO outperforms existing methods across various multi-agent interaction scenarios.
  arXiv  Detail & Related papers  (2025-10-06T15:07:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.