PAC-DP: PAC-Bayesian Diffusion Policy Learning

• URL: http://arxiv.org/abs/2607.24296v1
• Date: Mon, 27 Jul 2026 11:43:49 GMT
• Title: PAC-DP: PAC-Bayesian Diffusion Policy Learning
• Abstract summary: Diffusion Policies (DPs) are able to perform complex manipulation tasks.<n>We propose PAC-DP, an approach that increases the performance of DPs in robotic manipulation tasks.
• Score: 13.10166905442267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Policies (DPs) are able to perform complex manipulation tasks. However, DPs are typically trained by minimizing a denoising objective, which provides limited control over generalization in the finite-data regimes common in robotics. In this letter, we propose PAC-DP, an approach that increases the performance of DPs in robotic manipulation tasks. By modeling the DP as a Bayesian neural network, and defining a PAC-Bayes generalization bound, we derive a novel training objective that augments the standard denoising loss with a Kullback-Leibler divergence regularizer between the posterior and prior parameter distributions. From the theoretical perspective, our approach provides a principled approach to regularize the training of DPs without significantly increasing the training time. From the practical point of view, experimental results demonstrate improved denoising performance, lower variational negative log-likelihood, and higher success rates across multiple robotic manipulation benchmarks. Crucially, the largest improvements are observed in low-data training regimes and complex tasks, establishing PAC-DP as a theoretically grounded framework for robot policy learning.

Related papers

• Set-Supervised Diffusion Policy: Learning Action-Chunking Diffusion through Corrections [25.961145214027255]
  Diffusion policies provide paired supervision in the form of the robot's undesired actions and the human teacher's corrective actions.<n>We propose Set-Supervised Diffusion Policy (SDP), a novel learning framework that utilizes contrastive action-chunk data to train diffusion policies from human corrections.<n>SDP consistently improves policy performance, with particularly strong gains in robustness to noisy data.
  arXiv  Detail & Related papers  (2026-06-01T08:14:38Z)
• OGPO: Sample Efficient Full-Finetuning of Generative Control Policies [53.42266064673132]
  Generative control policies (GCPs) have emerged as effective parameterizations for robot learning.<n>This work introduces Off-policy Generative Policy Optimization (OGPO), a sample-efficient for finetuning GCPs.<n>OGPO achieves state-of-the-art performance on manipulation tasks spanning multi-task settings, high-precision insertion, and dexterous control.
  arXiv  Detail & Related papers  (2026-05-04T18:36:40Z)
• V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think [90.69263509098948]
  We introduce Variational GRPO, a method that integrates ELBO-based surrogates with the Group Relative Policy Optimization algorithm.<n>V-GRPO achieves state-of-the-art performance in text-to-image synthesis, while delivering a $2times$ speedup over MixGRPO and a $3times$ speedup over DiffusionNFT.
  arXiv  Detail & Related papers  (2026-04-25T17:03:21Z)
• Unbiased Dynamic Pruning for Efficient Group-Based Policy Optimization [60.87651283510059]
  Group Relative Policy Optimization (GRPO) effectively scales LLM reasoning but incurs prohibitive computational costs.<n>We propose Dynamic Pruning Policy Optimization (DPPO), a framework that enables dynamic pruning while preserving unbiased gradient estimation.<n>To mitigate the data sparsity induced by pruning, we introduce Dense Prompt Packing, a window-based greedy strategy.
  arXiv  Detail & Related papers  (2026-03-04T14:48:53Z)
• Decoupled Action Head: Confining Task Knowledge to Conditioning Layers [14.665308261922464]
  Behavior Cloning (BC) is a data-driven supervised learning approach.<n>We propose a decoupled training recipe that leverages observationfree data to pretrain a kinematics-generated action head.<n>Our experiments demonstrate the feasibility of this approach in both in-distribution and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-11-15T08:39:50Z)
• Fast Adaptation with Behavioral Foundation Models [82.34700481726951]
  Unsupervised zero-shot reinforcement learning has emerged as a powerful paradigm for pretraining behavioral foundation models.<n>Despite promising results, zero-shot policies are often suboptimal due to errors induced by the unsupervised training process.<n>We propose fast adaptation strategies that search in the low-dimensional task-embedding space of the pre-trained BFM to rapidly improve the performance of its zero-shot policies.
  arXiv  Detail & Related papers  (2025-04-10T16:14:17Z)
• Entropy Controllable Direct Preference Optimization [3.536605202672355]
  We propose a simple modification to DPO, H-DPO, which allows for control over the entropy of the resulting policy.<n>In our experiments, we show that H-DPO outperformed DPO across various tasks, demonstrating superior results in pass@$k$ evaluations for mathematical tasks.
  arXiv  Detail & Related papers  (2024-11-12T07:09:44Z)
• Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment [58.049113055986375]
  We develop a single stage approach named Alignment with Integrated Human Feedback (AIHF) to train reward models and the policy.<n>The proposed approach admits a suite of efficient algorithms, which can easily reduce to, and leverage, popular alignment algorithms.<n>We demonstrate the efficiency of the proposed solutions with extensive experiments involving alignment problems in LLMs and robotic control problems in MuJoCo.
  arXiv  Detail & Related papers  (2024-06-11T01:20:53Z)
• PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent [11.866227238721939]
  We propose a two-stage fine-tuning method, PAC-tuning, to address this optimization challenge. PAC-tuning directly minimizes the PAC-Bayes bound to learn proper parameter distribution. Second, PAC-tuning modifies the gradient by injecting noise with the variance learned in the first stage into the model parameters during training, resulting in a variant of perturbed descent.
  arXiv  Detail & Related papers  (2023-10-26T17:09:13Z)
• Improving Generalization of Complex Models under Unbounded Loss Using PAC-Bayes Bounds [10.94126149188336]
  PAC-Bayes learning theory has focused extensively on establishing tight upper bounds for test errors. A recently proposed training procedure called PAC-Bayes training, updates the model toward minimizing these bounds. This approach is theoretically sound, in practice, it has not achieved a test error as low as those obtained by empirical risk minimization (ERM) We introduce a new PAC-Bayes training algorithm with improved performance and reduced reliance on prior tuning.
  arXiv  Detail & Related papers  (2023-05-30T17:31:25Z)
• Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice [54.03076395748459]
  A central question in the meta-learning literature is how to regularize to ensure generalization to unseen tasks. We present a generalization bound for meta-learning, which was first derived by Rothfuss et al. We provide a theoretical analysis and empirical case study under which conditions and to what extent these guarantees for meta-learning improve upon PAC-Bayesian per-task learning bounds.
  arXiv  Detail & Related papers  (2022-11-14T08:51:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.