Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks

• URL: http://arxiv.org/abs/2512.07697v1
• Date: Mon, 08 Dec 2025 16:38:14 GMT
• Title: Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks
• Abstract summary: We introduce Delay-Aware Diffusion Policy (DA-DP), a framework for explicitly incorporating inference delays into policy learning.<n>DA-DP corrects zero-delay trajectories to their delay-compensated counterparts, and augments the policy with delay conditioning.<n>We empirically validate DA-DP on a variety of tasks, robots, and delays and find its success rate more robust to delay than delay-unaware methods.
• Score: 11.985096734655079
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As a robot senses and selects actions, the world keeps changing. This inference delay creates a gap of tens to hundreds of milliseconds between the observed state and the state at execution. In this work, we take the natural generalization from zero delay to measured delay during training and inference. We introduce Delay-Aware Diffusion Policy (DA-DP), a framework for explicitly incorporating inference delays into policy learning. DA-DP corrects zero-delay trajectories to their delay-compensated counterparts, and augments the policy with delay conditioning. We empirically validate DA-DP on a variety of tasks, robots, and delays and find its success rate more robust to delay than delay-unaware methods. DA-DP is architecture agnostic and transfers beyond diffusion policies, offering a general pattern for delay-aware imitation learning. More broadly, DA-DP encourages evaluation protocols that report performance as a function of measured latency, not just task difficulty.

Related papers

• Diffusion-Guided Uncertainty-Aware Delayed Policy Optimization [12.110304176149148]
  Reinforcement learning in real world environments often suffers from severe performance degradation due to delayed feedback.<n>We propose Diffusion Guided Uncertainty Aware Delayed Policy Optimization (DUPO)<n>Our method explicitly models the relationship between delayed state message and the current state using a diffusion model.
  arXiv  Detail & Related papers  (2026-07-06T13:33:17Z)
• Delay-Empowered Causal Hierarchical Reinforcement Learning [24.381499367239858]
  We propose Delay-Empowered Causal Hierarchical Reinforcement Learning (DECHRL)<n>DECHRL explicitly models both the causal structure of state transitions and their associated delay distributions.<n>We evaluate DECHRL in modified 2DMinecraft and MiniGrid environments featuring delays.
  arXiv  Detail & Related papers  (2026-05-12T15:28:32Z)
• When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making [68.12864562049957]
  Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
  arXiv  Detail & Related papers  (2026-03-17T15:38:50Z)
• DLLM Agent: See Farther, Run Faster [94.74432470237817]
  Diffusion large language models (DLLMs) have emerged as an alternative to autoregressive (AR) decoding with appealing efficiency and modeling properties.<n>We study this in a controlled setting by instantiatingDLLM and AR backbones within the same agent workflow.<n>We find thatDLLM Agents are on average over 30% faster end to end than AR agents, with some cases exceeding 8x speedup.
  arXiv  Detail & Related papers  (2026-02-07T09:01:18Z)
• TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration [64.32072516882947]
  Diffusion Policy excels in embodied control but suffers from high inference latency and computational cost.<n>We propose Temporal-aware Reinforcement-based Speculative Diffusion Policy (TS-DP)<n>TS-DP achieves up to 4.17 times faster inference with over 94% accepted drafts, reaching an inference frequency of 25 Hz.
  arXiv  Detail & Related papers  (2025-12-13T07:53:14Z)
• Adaptive Reinforcement Learning for Unobservable Random Delays [46.04329493317009]
  We introduce a general framework that enables agents to adaptively handle unobservable and time-varying delays.<n>Specifically, the agent generates a matrix of possible future actions to handle both unpredictable delays and lost action packets sent over networks.<n>Our method significantly outperforms state-of-the-art approaches across a wide range of benchmark environments.
  arXiv  Detail & Related papers  (2025-06-17T11:11:37Z)
• Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observation [10.511062258286335]
  In real-world multi-agent systems, observation delays are ubiquitous, preventing agents from making decisions based on the environment's true state.<n>These discrete observation components with varying delay characteristics pose significant challenges for multi-agent reinforcement learning.<n>We first formulate the decentralized individual delay partially observable decision process (DSID-POMDP) by extending the standard Dec-POMDP.<n>We then propose the Rainbow Delay Compensation (RDC) framework for addressing individual delays, along with recommended implementations for its constituent modules.
  arXiv  Detail & Related papers  (2025-05-06T14:47:56Z)
• DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays [26.032139258562708]
  We propose $textbfDEER (Delay-resilient-Enhanced RL)$, a framework designed to effectively enhance the interpretability and address the random delay issues. In a variety of delayed scenarios, the trained encoder can seamlessly integrate with standard RL algorithms without requiring additional modifications. The results confirm that DEER is superior to state-of-the-art RL algorithms in both constant and random delay settings.
  arXiv  Detail & Related papers  (2024-06-05T09:45:26Z)
• Tree Search-Based Policy Optimization under Stochastic Execution Delay [46.849634120584646]
  Delayed execution MDPs are a new formalism addressing random delays without resorting to state augmentation. We show that given observed delay values, it is sufficient to perform a policy search in the class of Markov policies. We devise DEZ, a model-based algorithm that optimize over the class of Markov policies.
  arXiv  Detail & Related papers  (2024-04-08T12:19:04Z)
• Delays in Reinforcement Learning [2.5835347022640254]
  This dissertation aims to study the delay in the agent's observation of the state of the environment or in the execution of the agent's actions. A wide spectrum of delays will be considered, and potential solutions will be presented.
  arXiv  Detail & Related papers  (2023-09-20T07:04:46Z)
• MTD: Multi-Timestep Detector for Delayed Streaming Perception [0.5439020425819]
  Streaming perception is a task of reporting the current state of the world, which is used to evaluate the delay and accuracy of autonomous driving systems. This paper propose the Multi- Timestep Detector (MTD), an end-to-end detector which uses dynamic routing for multi-branch future prediction. The proposed method has been evaluated on the Argoverse-HD dataset, and the experimental results show that it has achieved state-of-the-art performance across various delay settings.
  arXiv  Detail & Related papers  (2023-09-13T06:23:58Z)
• Reinforcement Learning with a Terminator [80.34572413850186]
  We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds. We use these to construct a provably-efficient algorithm, which accounts for termination, and bound its regret.
  arXiv  Detail & Related papers  (2022-05-30T18:40:28Z)
• Acting in Delayed Environments with Non-Stationary Markov Policies [57.52103323209643]
  We introduce a framework for learning and planning in MDPs where the decision-maker commits actions that are executed with a delay of $m$ steps. We prove that with execution delay, deterministic Markov policies in the original state-space are sufficient for attaining maximal reward, but need to be non-stationary. We devise a non-stationary Q-learning style model-based algorithm that solves delayed execution tasks without resorting to state-augmentation.
  arXiv  Detail & Related papers  (2021-01-28T13:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.