TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learning

• URL: http://arxiv.org/abs/2502.15425v4
• Date: Wed, 05 Mar 2025 10:48:42 GMT
• Title: TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learning
• Authors: Giuseppe Paolo, Abdelhakim Benechehab, Hamza Cherkaoui, Albert Thomas, Balázs Kégl,
• Abstract summary: We introduce TAME Agent Framework (TAG), a framework for constructing fully decentralized hierarchical multi-agent systems.<n>TAG standardizes information flow between levels while preserving loose coupling, allowing for seamless integration of diverse agent types.<n>Our results show that decentralized hierarchical organization enhances both learning speed and final performance, positioning TAG as a promising direction for scalable multi-agent systems.
• Score: 4.591755344464076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hierarchical organization is fundamental to biological systems and human societies, yet artificial intelligence systems often rely on monolithic architectures that limit adaptability and scalability. Current hierarchical reinforcement learning (HRL) approaches typically restrict hierarchies to two levels or require centralized training, which limits their practical applicability. We introduce TAME Agent Framework (TAG), a framework for constructing fully decentralized hierarchical multi-agent systems. TAG enables hierarchies of arbitrary depth through a novel LevelEnv concept, which abstracts each hierarchy level as the environment for the agents above it. This approach standardizes information flow between levels while preserving loose coupling, allowing for seamless integration of diverse agent types. We demonstrate the effectiveness of TAG by implementing hierarchical architectures that combine different RL agents across multiple levels, achieving improved performance over classical multi-agent RL baselines on standard benchmarks. Our results show that decentralized hierarchical organization enhances both learning speed and final performance, positioning TAG as a promising direction for scalable multi-agent systems.

Related papers

• MARTI-MARS$^2$: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation [64.2621682259008]
  Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2)<n>We propose a Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2) to integrate policy learning with multi-agent tree search.<n>We show that MARTI-MARS2 achieves 77.7%, outperforming strong baselines like GPT-5.1 on challenging code generation benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T07:28:44Z)
• Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism [61.01709143437043]
  We introduce a novel agent design framework centered on a Hierarchical Task Abstraction Mechanism (HTAM)<n>Specifically, HTAM moves beyond emulating social roles, instead structuring multi-agent systems into a logical hierarchy that mirrors the intrinsic task-dependency graph of a given domain.<n>We instantiate this framework as EarthAgent, a multi-agent system tailored for complex geospatial analysis.
  arXiv  Detail & Related papers  (2025-11-21T12:25:47Z)
• Minimizing Hyperbolic Embedding Distortion with LLM-Guided Hierarchy Restructuring [19.895748346987435]
  The quality of hyperbolic embeddings is tightly coupled to the structure of the input hierarchy.<n>This paper investigates whether Large Language Models (LLMs) have the ability to automatically restructure hierarchies to meet these criteria.<n> Experiments on 16 diverse hierarchies show that LLM-restructured hierarchies consistently yield higher-quality hyperbolic embeddings.
  arXiv  Detail & Related papers  (2025-11-16T18:10:20Z)
• A Semantics-Aware Hierarchical Self-Supervised Approach to Classification of Remote Sensing Images [12.282079123411947]
  We present a novel Semantics-Aware Hierarchical Consensus (SAHC) method for learning hierarchical features and relationships.<n>The SAHC method is evaluated on three benchmark datasets with different degrees of hierarchical complexity.<n> Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks.
  arXiv  Detail & Related papers  (2025-10-06T15:30:39Z)
• Feature Identification for Hierarchical Contrastive Learning [7.655211354400059]
  We propose two novel hierarchical contrastive learning (HMLC) methods.<n>Our approach explicitly models inter-class relationships and imbalanced class distribution at higher hierarchy levels.<n>Our method achieves state-of-the-art performance in linear evaluation, outperforming existing hierarchical contrastive learning methods by 2 percentage points in terms of accuracy.
  arXiv  Detail & Related papers  (2025-10-01T12:46:47Z)
• Diagnose, Localize, Align: A Full-Stack Framework for Reliable LLM Multi-Agent Systems under Instruction Conflicts [75.20929587906228]
  Large Language Model (LLM)-powered multi-agent systems (MAS) have rapidly advanced collaborative reasoning, tool use, and role-specialized coordination in complex tasks.<n>However, reliability-critical deployment remains hindered by a systemic failure mode: hierarchical compliance under instruction conflicts.
  arXiv  Detail & Related papers  (2025-09-27T08:43:34Z)
• A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applications [0.0]
  Hierarchical multi-agent systems (HMAS) organize collections of agents into layered structures that help manage complexity and scale.<n>This paper proposes a multi-dimensional taxonomy for HMAS along five axes: control hierarchy, information flow, role and task delegation, temporal layering, and communication structure.
  arXiv  Detail & Related papers  (2025-08-18T07:36:33Z)
• Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning [19.739901034066587]
  We propose a novel and effective methodology for learning multi-agent hierarchies of message-passing policies.<n>Agents at lower levels in the hierarchy receive goals from the upper levels and exchange messages with neighboring agents at the same level.<n>Results on relevant benchmarks show that our method performs favorably compared to the state of the art.
  arXiv  Detail & Related papers  (2025-07-31T14:42:12Z)
• HieraRS: A Hierarchical Segmentation Paradigm for Remote Sensing Enabling Multi-Granularity Interpretation and Cross-Domain Transfer [3.510584282402788]
  HieraRS is a novel hierarchical interpretation paradigm that enables multi-granularity predictions.<n>It supports the efficient transfer of LCLU models to cross-domain tasks with heterogeneous tree-structured hierarchies.
  arXiv  Detail & Related papers  (2025-07-11T16:44:01Z)
• AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction [70.60422261117816]
  We propose a new framework that rethinks multi-agent coordination through a sequential structure rather than a graph structure.<n>Our method focuses on two key directions: (1) Next-Agent Prediction, which selects the most suitable agent role at each step, and (2) Next-Context Selection, which enables each agent to selectively access relevant information from any previous step.
  arXiv  Detail & Related papers  (2025-06-21T18:34:43Z)
• MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents [59.825725526176655]
  Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents. Existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. We introduce MultiAgentBench, a benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios.
  arXiv  Detail & Related papers  (2025-03-03T05:18:50Z)
• Multilevel Verification on a Single Digital Decentralized Distributed (DDD) Ledger [0.0]
  In regular DDD ledgers, only a single level of verification is available. In systems where hierarchy emerges naturally, the inclusion of hierarchy in the solution enables us to come up with a better solution. The paper will address all these issues, and provide a road map to trace the state of the system at any given time and probability of failure of the system.
  arXiv  Detail & Related papers  (2024-09-03T08:47:08Z)
• RL-GPT: Integrating Reinforcement Learning and Code-as-policy [82.1804241891039]
  We introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent. The slow agent analyzes actions suitable for coding, while the fast agent executes coding tasks. This decomposition effectively focuses each agent on specific tasks, proving highly efficient within our pipeline.
  arXiv  Detail & Related papers  (2024-02-29T16:07:22Z)
• Creating Multi-Level Skill Hierarchies in Reinforcement Learning [0.0]
  We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction.
  arXiv  Detail & Related papers  (2023-06-16T17:23:49Z)
• Hierarchical Task Network Planning for Facilitating Cooperative Multi-Agent Reinforcement Learning [33.70599981505335]
  We present SOMARL, a framework that uses prior knowledge to reduce the exploration space and assist learning. In SOMARL, agents are treated as part of the MARL environment, and symbolic knowledge is embedded using a tree structure to build a knowledge hierarchy. We evaluate SOMARL on two benchmarks, FindTreasure and MoveBox, and report superior performance over state-of-the-art MARL environments.
  arXiv  Detail & Related papers  (2023-06-14T08:51:43Z)
• Hierarchical Decentralized Deep Reinforcement Learning Architecture for a Simulated Four-Legged Agent [0.0]
  In nature, control of movement happens in a hierarchical and decentralized fashion. We present a novel decentral, hierarchical architecture to control a simulated legged agent.
  arXiv  Detail & Related papers  (2022-09-21T07:55:33Z)
• Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework [75.79736930414715]
  We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes. We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
  arXiv  Detail & Related papers  (2022-04-27T21:41:44Z)
• Label Hierarchy Transition: Delving into Class Hierarchies to Enhance Deep Classifiers [40.993137740456014]
  We propose a unified probabilistic framework based on deep learning to address the challenges of hierarchical classification. The proposed framework can be readily adapted to any existing deep network with only minor modifications. We extend our proposed LHT framework to the skin lesion diagnosis task and validate its great potential in computer-aided diagnosis.
  arXiv  Detail & Related papers  (2021-12-04T14:58:36Z)
• Provable Hierarchy-Based Meta-Reinforcement Learning [50.17896588738377]
  We analyze HRL in the meta-RL setting, where learner learns latent hierarchical structure during meta-training for use in a downstream task. We provide "diversity conditions" which, together with a tractable optimism-based algorithm, guarantee sample-efficient recovery of this natural hierarchy. Our bounds incorporate common notions in HRL literature such as temporal and state/action abstractions, suggesting that our setting and analysis capture important features of HRL in practice.
  arXiv  Detail & Related papers  (2021-10-18T17:56:02Z)
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression [53.90578309960526]
  Large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. We propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
  arXiv  Detail & Related papers  (2021-10-16T11:23:02Z)
• HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism [17.993973801986677]
  Multi-agent reinforcement learning often suffers from the exponentially larger action space caused by a large number of agents. We propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for the fully cooperative multi-agent problems.
  arXiv  Detail & Related papers  (2021-10-14T10:43:47Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.