HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation

• URL: http://arxiv.org/abs/2606.30966v1
• Date: Mon, 29 Jun 2026 23:03:05 GMT
• Title: HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation
• Abstract summary: This paper introduces HyPOLE, a novel framework for Multi-Agent Reinforcement Learning (MARL) under partial observability.<n>We integrate Hyper Training for Decentralized Execution (CTDE) techniques with HyPOLE to synthesize decentralized policies.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives. However, these benefits remain largely unexplored in the context of Multi-Agent Reinforcement Learning (MARL). This paper introduces HyPOLE, a novel framework for MARL under partial observability, where learning is guided by the expressive power of the so-called hyperproperties and, in particular, the temporal logic HyperLTL. We integrate Centralized Training for Decentralized Execution (CTDE) techniques with HyPOLE to synthesize decentralized policies, and our evaluation on SMAC, MessySMAC, and WildFire benchmark demonstrates clear advantages over baselines.

Related papers

• UniER: A Unified Benchmark for Item-level and Path-level Exercise Recommendation [53.62147575168325]
  We present a Unified Benchmark for Exercise Recommendation (UniER)<n>UniER is a comprehensive evaluation framework that unifies ILER and PLER.<n>Our results reveal the systematic dominance of PLER and expose the pedagogical failure of ILER's fragmented recommendations.
  arXiv  Detail & Related papers  (2026-05-16T02:07:58Z)
• Preference-Guided Learning for Sparse-Reward Multi-Agent Reinforcement Learning [15.034714081414691]
  We study the problem of online multi-agent reinforcement learning (MARL) in environments with sparse rewards.<n>The lack of intermediate rewards hinders standard MARL algorithms from effectively guiding policy learning.<n>We propose a novel framework that integrates online inverse preference learning with multi-agent on-policy optimization.
  arXiv  Detail & Related papers  (2025-09-26T03:41:40Z)
• Learning to Deliberate: Meta-policy Collaboration for Agentic LLMs with Multi-agent Reinforcement Learning [13.9097452260769]
  We introduce the Meta-Policy Deliberation Framework (MPDF), where agents learn a decentralized policy over a set of high-level meta-cognitive actions.<n>We develop SoftRankPO, a novel reinforcement learning algorithm.
  arXiv  Detail & Related papers  (2025-09-04T02:06:06Z)
• The Landscape of Agentic Reinforcement Learning for LLMs: A Survey [103.32591749156416]
  The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL)<n>This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL.
  arXiv  Detail & Related papers  (2025-09-02T17:46:26Z)
• Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learning [106.68304931854038]
  Reinforcement learning with verifiable rewards (RLVR) has been widely used for enhancing the reasoning abilities of large language models (LLMs)<n>We conduct a systematic empirical analysis of the entropy-performance exchange mechanism of RLVR across different levels of granularity.<n>Our analysis reveals that, in the rising stage, entropy reduction in negative samples facilitates the learning of effective reasoning patterns.<n>In the plateau stage, learning efficiency strongly correlates with high-entropy tokens present in low-perplexity samples and those located at the end of sequences.
  arXiv  Detail & Related papers  (2025-08-04T10:08:10Z)
• Training Large Language Models to Reason via EM Policy Gradient [0.27195102129094995]
  We introduce an off-policy reinforcement learning algorithm, EM Policy Gradient, to enhance LLM reasoning.<n>We evaluate the effectiveness of EM Policy Gradient on the GSM8K and MATH (HARD) datasets.<n>Models fine-tuned with our method exhibit cognitive behaviors, such as sub-problem decomposition, self-verification, and backtracking.
  arXiv  Detail & Related papers  (2025-04-24T01:31:05Z)
• HypRL: Reinforcement Learning of Control Policies for Hyperproperties [0.0]
  We propose HYPRL, a specification-guided reinforcement learning framework.<n>We apply Skolemization to manage quantifier alternations and define quantitative functions to shape rewards.<n>A suitable RL algorithm is then used to learn policies that collectively maximize the expected reward.
  arXiv  Detail & Related papers  (2025-04-07T01:58:36Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training [8.419845742978985]
  We investigate the challenging framework of Byzantine-robust training in distributed machine learning (ML) systems. Our first contribution is the introduction of an efficient meta-aggregator that upgrades baseline aggregators to optimal performance levels. Our paper highlights its theoretical and practical advantages for Byzantine-robust training, especially in simplifying the tuning process.
  arXiv  Detail & Related papers  (2024-05-23T16:29:30Z)
• MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learning [128.19212716007794]
  We propose an effective framework called textbfMulti-textbfAgent textbfMasked textbfAttentive textbfContrastive textbfLearning (MA2CL) MA2CL encourages learning representation to be both temporal and agent-level predictive by reconstructing the masked agent observation in latent space. Our method significantly improves the performance and sample efficiency of different MARL algorithms and outperforms other methods in various vision-based and state-based scenarios.
  arXiv  Detail & Related papers  (2023-06-03T05:32:19Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.