Diffusion Policies for Short-Horizon Planning in Robot Crowd Navigation

• URL: http://arxiv.org/abs/2608.27158v1
• Date: Thu, 27 Aug 2026 14:10:39 GMT
• Title: Diffusion Policies for Short-Horizon Planning in Robot Crowd Navigation
• Abstract summary: Planning Diffusion Policy Optimization generates short-horizon action chunks for crowd navigation.<n>We introduce a setting in which boundary violations are treated as collisions.<n>Experiments show that PDPO obtains an improved success rate over strong baselines.
• Score: 2.9434930072968584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robot crowd navigation requires safe and efficient decision-making under dense, dynamic, and multimodal human--robot interactions. Existing reinforcement-learning methods typically output a single reactive action at each timestep, which limits their ability to represent diverse short-term avoidance strategies. We propose Planning Diffusion Policy Optimization (PDPO), an offline-to-online reinforcement-learning framework that uses a diffusion policy to generate short-horizon action chunks for crowd navigation. PDPO is first pretrained on collision-avoidance demonstrations and then fine-tuned online with PPO by treating the denoising process as an internal decision process. During execution, the policy generates a five-step action chunk and applies it in a receding-horizon manner. Furthermore, we observe an evaluation artifact in common crowd-navigation benchmarks: without explicit boundary constraints, learned agents may leave the valid domain and bypass dense crowds. To address this, we introduce a setting in which boundary violations are treated as collisions. Experiments show that PDPO obtains an improved success rate over strong baselines, and ablations demonstrate that action chunks are especially important for the modified bounded benchmark.

Related papers

• From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training [55.744284393275926]
  Reinforcement learning (RL) has become a widely adopted technique for improving large language models (LLMs) on complex tasks.<n>Existing RL methods still face challenges in training agents with longer-horizon interactions.<n>We introduce a novel policy gradient method, Hindsight Policy Optimization (HPO), that projects both the current policy distribution and the hindsight distribution into an intent space.
  arXiv  Detail & Related papers  (2026-06-28T08:35:47Z)
• Temporal Logic Guidance for Action-Only Diffusion Policies with World Models [0.764671395172401]
  Diffusion policies enable multimodal robot behavior but offer limited ability to choose among behavior modes at inference time.<n>We propose a novel guidance method for action-only diffusion policies that uses a separate learned world model to enable differentiable evaluation of STL.<n>This steers behavior toward constraint satisfaction without retraining, improving constraint adherence while preserving task performance.
  arXiv  Detail & Related papers  (2026-06-22T00:12:44Z)
• Aligning Flow Map Policies with Optimal Q-Guidance [50.514994916864275]
  Flow map policies are designed for fast action generation by learning to take arbitrary-size jumps.<n>We derive FLOW MAP Q-GUIDANCE (FMQ), a principled closed-form learning target that is optimal for adapting offline flow map policies under a critic-guided trust-region constraint.<n>FMQ achieves state-of-the-art performance in offline-to-online RL, outperforming the previous one-step policy MVP by a relative improvement of 21.3% on the average success rate.
  arXiv  Detail & Related papers  (2026-05-12T17:12:29Z)
• Generative Actor-Critic with Soft Bridge Policies [19.121990264725028]
  Training effective soft generative policies faces two obstacles that often arise together.<n>First, marginal action densities are often unavailable, so existing methods typically rely on entropy bounds, proxies or approximations.<n>These obstacles motivate us to seek a generative policy that exposes a tractable MaxEnt objective while requiring only a single sampled actor forward pass for action generation.
  arXiv  Detail & Related papers  (2026-05-09T06:36:32Z)
• AIPO: Learning to Reason from Active Interaction [54.10819421625103]
  AIPO enables the policy model to proactively consult three functional collaborative agents when encountering reasoning bottlenecks.<n>AIPO consistently improves reasoning performance, generalizes robustly across different policy models and RLVR algorithms, and effectively expands the reasoning capability boundary of the policy model.
  arXiv  Detail & Related papers  (2026-05-08T19:06:55Z)
• V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think [90.69263509098948]
  We introduce Variational GRPO, a method that integrates ELBO-based surrogates with the Group Relative Policy Optimization algorithm.<n>V-GRPO achieves state-of-the-art performance in text-to-image synthesis, while delivering a $2times$ speedup over MixGRPO and a $3times$ speedup over DiffusionNFT.
  arXiv  Detail & Related papers  (2026-04-25T17:03:21Z)
• From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation [18.70033095161235]
  We propose a framework that distills a Conditional Flow Matching expert into a fast single-step student via Implicit Likelihood Estimation (IMLE)<n>A bi-directional Chamfer distance provides a set-level objective that promotes both mode coverage and fidelity.<n>A unified perception encoder further integrates multi-view RGB, depth, point clouds, and proprioception into a geometry-aware representation.
  arXiv  Detail & Related papers  (2026-03-10T09:30:05Z)
• Latent Policy Steering through One-Step Flow Policies [34.06099184809882]
  offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration.<n>Latent Policy Steering (LPS) enables high-fidelity latent policy improvement by backpropagating original-action-space Q-gradients through a differentiable one-step MeanFlow policy.<n>Across OGBench and real-world robotic tasks, LPS achieves state-of-the-art performance and consistently outperforms behavioral cloning and strong latent steering baselines.
  arXiv  Detail & Related papers  (2026-03-05T15:38:08Z)
• Coverage Improvement and Fast Convergence of On-policy Preference Learning [67.36750525893514]
  Online on-policy preference learning algorithms for language model alignment can significantly outperform their offline counterparts.<n>We analyze how the sampling policy's coverage evolves throughout on-policy training.<n>We develop principled on-policy schemes for reward distillation in the general function class setting.
  arXiv  Detail & Related papers  (2026-01-13T10:46:06Z)
• CAPE: Context-Aware Diffusion Policy Via Proximal Mode Expansion for Collision Avoidance [15.311155448797386]
  contexts-aware diffusion policy via Proximal mode Expansion (CAPE)<n>CAPE expands trajectory distribution modes with context-aware prior and guidance at inference.<n>We evaluate CAPE on diverse manipulation tasks in cluttered unseen simulated and real-world settings.
  arXiv  Detail & Related papers  (2025-11-27T21:53:09Z)
• SDPO: Segment-Level Direct Preference Optimization for Social Agents [56.970902914217156]
  Social agents powered by large language models (LLMs) can simulate human social behaviors but fall short in handling complex social dialogues.<n>We propose Segment-Level Direct Preference Optimization (SDPO), which dynamically select key segments within interactions to optimize multi-turn agent behavior.
  arXiv  Detail & Related papers  (2025-01-03T14:09:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.