Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering

• URL: http://arxiv.org/abs/2605.26286v1
• Date: Mon, 25 May 2026 19:19:46 GMT
• Title: Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
• Abstract summary: Real-world multi-agent reinforcement learning (MARL) systems must often operate under stale observations, communication delays, and intermittent packet loss.<n>We propose a modular execution-stage state-estimation layer that replaces delayed communicated observations with current belief-state estimates.
• Score: 2.0223202343873226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world multi-agent reinforcement learning (MARL) systems must often operate under stale observations, stochastic communication delays, and intermittent packet loss. Policies trained under idealized synchronous conditions frequently exhibit significant performance degradation in these regimes because they act on outdated feedback. We propose a modular execution-stage state-estimation layer that replaces delayed communicated observations with current belief-state estimates. The framework integrates a learned Gated transition model with a recursive Kalman filtering layer to estimate instantaneous states from asynchronous measurements. A primary advantage of this approach is its modularity, The estimator serves as a plug-in for pre-trained policies, requiring no modifications to the original MARL training algorithm, architecture, or reward structure. Evaluation across diverse multi-agent and continuous-control benchmarks demonstrates that the proposed layer consistently enhances robustness to communication latency and message loss. The most significant performance gains are observed in coordination-intensive and dynamically unstable tasks where temporal consistency is critical for control.

Related papers

• Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning [17.62926868585848]
  Cooperative Multi-Agent Reinforcement Learning (MARL) frequently suffers from severe reward sparsity and exploration bottlenecks.<n>We propose Episodic Memory Temporal Consistency (EMTC), a framework that robustly constructs and selectively leverages historical experiences.<n>EMTC achieves absolute win-rate improvements of up to 24% in super-hard SMAC scenarios and an average improvement of 28% across GRF tasks.
  arXiv  Detail & Related papers  (2026-06-03T06:15:39Z)
• Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling [11.212756680882444]
  We introduce RACE-Sched, an agent-based framework that decouples policy execution from logical reasoning via a dual-stream architecture.<n>This approach harmonizes real-time constraints with long-horizon reasoning to achieve superior solution quality and robust adaptation to dynamic events.
  arXiv  Detail & Related papers  (2026-05-28T02:26:18Z)
• Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index [51.56484100374058]
  We propose an online, topology-aware monitor for evolving neural representations that couples Modular Morse Homology Maintenance (MMHM) with a composite Collapse Index (CI)<n>Instead of rebuilding complexes each epoch, we apply sparse edits at a fixed scale and maintain a discrete Morse matching, yielding fast, incremental updates.
  arXiv  Detail & Related papers  (2026-04-28T19:21:21Z)
• Delayed Homomorphic Reinforcement Learning for Environments with Delayed Feedback [11.866061471514582]
  Delayed feedback breaks the Markov assumption and impedes learning and control.<n>We propose a framework grounded in MDP homomorphisms that collapses belief-equivalent augmented states.<n> Experiments on continuous control tasks in MuJoCo benchmark confirm that our algorithm outperforms strong augmentation-based baselines.
  arXiv  Detail & Related papers  (2026-04-04T08:38:52Z)
• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift [64.37959940809633]
  We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures.<n>We show Transformer-based sequence policies substantially outperform, RNN, and SSMs in robustness, maintaining high returns even when large fractions of sensors are unavailable.
  arXiv  Detail & Related papers  (2026-03-04T22:21:54Z)
• A Representation-Consistent Gated Recurrent Framework for Robust Medical Time-Series Classification [0.0]
  We propose a representation-consistent gated recurrent framework (RC-GRF) that introduces a principled regularization strategy to enforce temporal consistency in hidden-state representations.<n>The proposed framework is model-agnostic and can be integrated into existing gated recurrent architectures without modifying their internal gating mechanisms.
  arXiv  Detail & Related papers  (2026-02-10T17:16:49Z)
• DFPO: Scaling Value Modeling via Distributional Flow towards Robust and Generalizable LLM Post-Training [94.568675548967]
  Training reinforcement learning (RL) systems in real-world environments remains challenging due to noisy supervision and poor out-of-domain generalization.<n>Recent distributional RL methods improve robustness by modeling values with multiple quantile points, but they still learn each quantile independently as a scalar.<n>We propose DFPO, a robust distributional RL framework that models values as continuous flows across time steps.
  arXiv  Detail & Related papers  (2026-02-05T17:07:42Z)
• DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training [39.30046923897652]
  We propose a Dependency Controlled Pre-training framework that explicitly models dynamic, multi-scale dependencies by simulating evolving inter-patch dependencies.<n>DeCoP achieves state-of-the-art results on ten datasets with lower computing resources, improving MSE by 3% on ETTh1 over PatchTST using only 37% of the FLOPs.
  arXiv  Detail & Related papers  (2025-09-18T05:44:06Z)
• Learning Unified System Representations for Microservice Tail Latency Prediction [8.532290784939967]
  Microservice architectures have become the de facto standard for building scalable cloud-native applications.<n>Traditional approaches often rely on per-request latency metrics, which are highly sensitive to transient noise.<n>We propose USRFNet, a deep learning network that explicitly separates and models traffic-side and resource-side features.
  arXiv  Detail & Related papers  (2025-08-03T07:46:23Z)
• WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training [64.0932926819307]
  We present Warmup-Stable and Merge (WSM), a framework that establishes a formal connection between learning rate decay and model merging.<n>WSM provides a unified theoretical foundation for emulating various decay strategies.<n>Our framework consistently outperforms the widely-adopted Warmup-Stable-Decay (WSD) approach across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-07-23T16:02:06Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning [53.18060442931179]
  We propose the age of semantics (AoS) for measuring semantics freshness of status updates in a cooperative relay communication system. We derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset.
  arXiv  Detail & Related papers  (2022-09-19T11:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.