What Makes a Good Diffusion Planner for Decision Making?

• URL: http://arxiv.org/abs/2503.00535v1
• Date: Sat, 01 Mar 2025 15:31:14 GMT
• Title: What Makes a Good Diffusion Planner for Decision Making?
• Authors: Haofei Lu, Dongqi Han, Yifei Shen, Dongsheng Li,
• Abstract summary: We train and evaluate over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy.<n>We reveal that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance.
• Score: 31.743124638746558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.

Related papers

• Latent Diffusion Planning for Imitation Learning [78.56207566743154]
  Latent Diffusion Planning (LDP) is a modular approach consisting of a planner and inverse dynamics model. By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data. On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches.
  arXiv  Detail & Related papers  (2025-04-23T17:53:34Z)
• Habitizing Diffusion Planning for Efficient and Effective Decision Making [41.128266491447334]
  We introduce Habi, a framework that transforms powerful but slow diffusion planning models into fast decision-making models.<n>Even using a laptop CPU, the habitized model can achieve an average 800+ Hz decision-making frequency.
  arXiv  Detail & Related papers  (2025-02-10T12:40:32Z)
• Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets [65.42834731617226]
  We propose a reinforcement learning method for diffusion model finetuning, dubbed Nabla-GFlowNet. We show that our proposed method achieves fast yet diversity- and prior-preserving finetuning of Stable Diffusion, a large-scale text-conditioned image diffusion model.
  arXiv  Detail & Related papers  (2024-12-10T18:59:58Z)
• Improved Noise Schedule for Diffusion Training [51.849746576387375]
  We propose a novel approach to design the noise schedule for enhancing the training of diffusion models.<n>We empirically demonstrate the superiority of our noise schedule over the standard cosine schedule.
  arXiv  Detail & Related papers  (2024-07-03T17:34:55Z)
• Diffusion Models in Low-Level Vision: A Survey [82.77962165415153]
  diffusion model-based solutions have emerged as widely acclaimed for their ability to produce samples of superior quality and diversity.<n>We present three generic diffusion modeling frameworks and explore their correlations with other deep generative models.<n>We summarize extended diffusion models applied in other tasks, including medical, remote sensing, and video scenarios.
  arXiv  Detail & Related papers  (2024-06-17T01:49:27Z)
• DiffuserLite: Towards Real-time Diffusion Planning [39.93614402208524]
  We introduce a super fast and lightweight diffusion planning framework, which employs a planning refinement process (PRP) to generate coarse-to-fine-grained trajectories. Our experimental results demonstrate that diffuserLite achieves a decision-making frequency of 122.2Hz and reaches state-of-the-art performance on D4RL, Robomimic, and FinRL benchmarks. In addition, diffuserLite can also serve as a flexible plugin to increase the decision-making frequency of other diffusion planning algorithms.
  arXiv  Detail & Related papers  (2024-01-27T15:30:49Z)
• Simple Hierarchical Planning with Diffusion [54.48129192534653]
  Diffusion-based generative methods have proven effective in modeling trajectories with offline datasets. We introduce the Hierarchical diffuser, a fast, yet surprisingly effective planning method combining the advantages of hierarchical and diffusion-based planning. Our model adopts a "jumpy" planning strategy at the higher level, which allows it to have a larger receptive field but at a lower computational cost.
  arXiv  Detail & Related papers  (2024-01-05T05:28:40Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Plans [25.326624139426514]
  Diffusion-based planning has shown promising results in long-horizon, sparse-reward tasks. However, due to their nature as generative models, diffusion models are not guaranteed to generate feasible plans. We propose a novel approach to refine unreliable plans generated by diffusion models by providing refining guidance to error-prone plans.
  arXiv  Detail & Related papers  (2023-10-30T10:35:42Z)
• Phasic Content Fusing Diffusion Model with Directional Distribution Consistency for Few-Shot Model Adaption [73.98706049140098]
  We propose a novel phasic content fusing few-shot diffusion model with directional distribution consistency loss. Specifically, we design a phasic training strategy with phasic content fusion to help our model learn content and style information when t is large. Finally, we propose a cross-domain structure guidance strategy that enhances structure consistency during domain adaptation.
  arXiv  Detail & Related papers  (2023-09-07T14:14:11Z)
• On the Design Fundamentals of Diffusion Models: A Survey [9.183452635904278]
  We organize this review according to their three key components, namely the forward process, the reverse process, and the sampling procedure. This allows us to provide a fine-grained perspective of diffusion models, benefiting future studies in the analysis of individual components, the applicability of design choices, and the implementation of diffusion models.
  arXiv  Detail & Related papers  (2023-06-07T15:46:47Z)
• Planning with Diffusion for Flexible Behavior Synthesis [125.24438991142573]
  We consider what it would look like to fold as much of the trajectory optimization pipeline as possible into the modeling problem. The core of our technical approach lies in a diffusion probabilistic model that plans by iteratively denoising trajectories.
  arXiv  Detail & Related papers  (2022-05-20T07:02:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.