Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation

• URL: http://arxiv.org/abs/2601.22965v1
• Date: Fri, 30 Jan 2026 13:27:59 GMT
• Title: Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation
• Abstract summary: Self-Imitated Diffusion Policy (SIDP) is a novel framework that learns improved planning by selectively imitating a set of trajectories sampled from itself.<n>Specifically, SIDP introduces a reward-guided self-imitation mechanism that encourages the policy to consistently produce high-quality trajectories efficiently.
• Score: 7.341858898582114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion policies (DP) have demonstrated significant potential in visual navigation by capturing diverse multi-modal trajectory distributions. However, standard imitation learning (IL), which most DP methods rely on for training, often inherits sub-optimality and redundancy from expert demonstrations, thereby necessitating a computationally intensive "generate-then-filter" pipeline that relies on auxiliary selectors during inference. To address these challenges, we propose Self-Imitated Diffusion Policy (SIDP), a novel framework that learns improved planning by selectively imitating a set of trajectories sampled from itself. Specifically, SIDP introduces a reward-guided self-imitation mechanism that encourages the policy to consistently produce high-quality trajectories efficiently, rather than outputs of inconsistent quality, thereby reducing reliance on extensive sampling and post-filtering. During training, we employ a reward-driven curriculum learning paradigm to mitigate inefficient data utility, and goal-agnostic exploration for trajectory augmentation to improve planning robustness. Extensive evaluations on a comprehensive simulation benchmark show that SIDP significantly outperforms previous methods, with real-world experiments confirming its effectiveness across multiple robotic platforms. On Jetson Orin Nano, SIDP delivers a 2.5$\times$ faster inference than the baseline NavDP, i.e., 110ms VS 273ms, enabling efficient real-time deployment.

Related papers

• DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models [55.01951088768769]
  DiffusionOPD is a new multi-task training paradigm for diffusion models based on Online Policy Distillation (OPD)<n>We show that DiffusionOPD consistently surpasses both multi-reward RL and cascade RL baselines in training efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-14T16:49:09Z)
• TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment [52.570581883709345]
  We propose Trajectory Matching Policy Balance Optimization (TMPO), which replaces reward with human-level reward distribution matching.<n>TMPO improves generative diversity over state-of-the-art methods by 9.1%, and achieves competitive performance in all downstream and efficiency metrics.<n>To further reduce multi-trajectory training time on large-scale flow- prefixes, TMPO incorporates Dynamic Tree Sampling models, where trajectories share denoising and branch at dynamically scheduled steps.
  arXiv  Detail & Related papers  (2026-05-09T04:41:02Z)
• E$^2$DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation [12.326967455610536]
  Decision Transformer (DT) has emerged as an effective framework for addressing long-horizon tasks.<n>E$2$DT is a DT-guided k-Determinantal Point Process sampling framework.<n>Our framework is experience-aware, allowing E$2$DT to be both efficient.
  arXiv  Detail & Related papers  (2026-04-30T19:28:44Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models [19.819472980239826]
  We develop a robust, scalable reward model based on a state-of-the-art VLM.<n>We employ these VLM rewards to guide the model to correct sub-optimal behaviors in a closed-loop manner.
  arXiv  Detail & Related papers  (2026-03-17T02:22:16Z)
• Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models [49.04912820721943]
  Reinforcement learning (RL) finetuning has become a key technique for enhancing the reasoning abilities of large language models (LLMs)<n>Recent advances underscore the importance of online prompt selection methods, which typically concentrate training on partially solved or moderately challenging examples.<n>This work proposes Dynamics-Predictive Sampling (DPS), which online predicts and selects informative prompts by inferring their learning dynamics prior to costly rollouts.
  arXiv  Detail & Related papers  (2026-03-11T15:31:14Z)
• PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning [17.21182051618103]
  Diffusion-based planners have emerged as a promising approach for human-like trajectory generation in autonomous driving.<n>Recent works incorporate reinforcement fine-tuning to enhance the robustness of diffusion planners.<n>We propose PlannerRFT, a sample-efficient reinforcement fine-tuning framework for diffusion-based planners.
  arXiv  Detail & Related papers  (2026-01-19T09:56:05Z)
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward [54.708851958671794]
  We propose a Data-Efficient Policy Optimization pipeline that combines optimized strategies for both offline and online data selection.<n>In offline phase, we curate a high-quality subset of training samples based on diversity, influence, and appropriate difficulty.<n>During online RLVR training, we introduce a sample-level explorability metric to dynamically filter samples with low exploration potential.
  arXiv  Detail & Related papers  (2025-09-01T10:04:20Z)
• Fake it till You Make it: Reward Modeling as Discriminative Prediction [49.31309674007382]
  GAN-RM is an efficient reward modeling framework that eliminates manual preference annotation and explicit quality dimension engineering.<n>Our method trains the reward model through discrimination between a small set of representative, unpaired target samples.<n>Experiments demonstrate our GAN-RM's effectiveness across multiple key applications.
  arXiv  Detail & Related papers  (2025-06-16T17:59:40Z)
• Prior-Guided Diffusion Planning for Offline Reinforcement Learning [5.819784482811376]
  Prior Guidance (PG) is a novel guided sampling framework that replaces the standard Gaussian prior of a behavior-cloned diffusion model.<n>We present an efficient training strategy that applies behavior regularization in latent space, and empirically demonstrate that PG outperforms state-the-art diffusion policies and planners across diverse long-horizon offline RL benchmarks.
  arXiv  Detail & Related papers  (2025-05-16T05:39:02Z)
• HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection [66.42229859018775]
  We introduce a unified, high-capacity weakly supervised object detection (WSOD) network called HUWSOD. HUWSOD incorporates a self-supervised proposal generator and an autoencoder proposal generator with a multi-rate re-supervised pyramid to replace traditional object proposals. Our findings indicate that randomly boxes, although significantly different from well-designed offline object proposals, are effective for WSOD training.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.