Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

• URL: http://arxiv.org/abs/2606.24208v1
• Date: Tue, 23 Jun 2026 06:49:30 GMT
• Title: Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control
• Abstract summary: For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible.<n>We propose an inference-time optimization framework that couples the behavior generation to physical feasibility.<n>We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints.
• Score: 41.782471938706344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible, violating reachability, collision-avoidance, or closed-loop executability requirements. This embodiment gap limits zero-shot deployment across robots, even when the task-space behavior itself is transferable. We propose an inference-time optimization framework that couples the behavior generation to physical feasibility by formulating diffusion guidance as a constrained optimization problem. Our key insight is to replace the sampling perturbation in the backward process with an optimized correction, allowing hard constraints or soft penalties to be imposed during sampling without the need to retrain the diffusion model, while keeping samples close to the learned prior. We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints. Across settings and robot embodiments, optimization-guided denoising matches the feasibility of projection- and gradient-guidance baselines while better preserving grasp quality, and improving controller-level executability and task success, with task success improving by up to 20pp. on dexterous grasping and 23pp. on visuomotor manipulation over the best baseline.

Related papers

• Motion Planning with Model-Based Diffusion via Constraint Optimization and Adaptive Scheduling [7.761673975696802]
  Single-Robot Motion Planning (SRMP) is challenging under complex constraints and computational limits.<n>Recent Model-Based Diffusion (MBD) recasts the SRMP to guide samples from a posterior diffusion trajectories.<n>Our method achieves higher safety & success rates, faster convergence, and lower final costs than baseline planners.
  arXiv  Detail & Related papers  (2026-07-16T01:04:32Z)
• From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation [108.02425846269234]
  controllable image generation has made significant strides by incorporating visual reference conditions.<n>Existing methods predominantly operate as open-loop systems.<n>We propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem.
  arXiv  Detail & Related papers  (2026-07-06T05:36:30Z)
• DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising [4.249024052507976]
  We introduce DiRecT, a training-free algorithm for constrained sampling from diffusion models via optimal control (SOC)<n>Inspired by model predictive control, we derive a receding-horizon surrogate for the otherwise intractable constrained SOC formulation.<n>Experiments demonstrate that DiRecT substantially improves deployment safety and task performance over existing diffusion-based planning baselines.
  arXiv  Detail & Related papers  (2026-06-13T15:41:26Z)
• Manifold-Constrained MPPI: Real-Time Sampling-Based Control Under Hard Constraints [8.747309323431802]
  Manifold-Constrained MPPI (MC-MPPI) is a real-time sampling-based control framework.<n>It enforces manifold-based equality constraints while preserving the computational advantages of MPPI.<n>Experiments on a 14-DoF closed-chain dual-arm system in both simulation and real-world settings demonstrate that MC-MPPI operates stably at 100 Hz.
  arXiv  Detail & Related papers  (2026-05-24T01:57:18Z)
• Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space [99.30250506439678]
  This work reformulates language generation as an optimal control problem.<n>It provides a unified theoretical perspective to analyze autoregressive and diffusion models.
  arXiv  Detail & Related papers  (2026-05-14T08:13:43Z)
• Sampling-Based Control via Entropy-Regularized Optimal Transport [18.278741443124655]
  OT-MPC is a sampling-based algorithm that overcomes limitations through an entropy-regularized optimal transport formulation.<n>We derive closed-form, gradient-free updates via the Sinkhorn algorithm, enabling real-time performance.
  arXiv  Detail & Related papers  (2026-05-04T02:07:15Z)
• Adaptive Linear Path Model-Based Diffusion [52.84663832658799]
  We introduce Linear Path Model-Based Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching linear probability path.<n>We also propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust diffusion steps and noise levels according to task complexity and environmental conditions.
  arXiv  Detail & Related papers  (2026-02-02T21:33:03Z)
• Warm-Starting Collision-Free Model Predictive Control With Object-Centric Diffusion [38.363402738466334]
  We show that combining diffusion-based warm-starting conditioned with a latent object-centric representation of the scene yields reliable and efficient motion generation under strict time limits.<n>Our approach conditions a diffusion transformer on the system state, task, and surroundings, using an object-centric slot attention mechanism.<n>The sampled trajectories are refined by an optimal control problem that enforces rigid-body dynamics and signed-distance collision constraints.
  arXiv  Detail & Related papers  (2026-01-06T10:02:54Z)
• HardFlow: Hard-Constrained Sampling for Flow-Matching Models via Trajectory Optimization [4.249024052507976]
  We introduce a novel framework that reformulates hard-constrained sampling as a trajectory optimization problem.<n>Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time.<n>Our algorithm, which we name $textitHardFlow$, substantially outperforms existing methods in both constraint satisfaction and sample quality.
  arXiv  Detail & Related papers  (2025-11-11T16:33:57Z)
• Diffusion Predictive Control with Constraints [51.91057765703533]
  Diffusion predictive control with constraints (DPCC) is an algorithm for diffusion-based control with explicit state and action constraints.<n>We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints.
  arXiv  Detail & Related papers  (2024-12-12T15:10:22Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.