Diffusion Predictive Control with Constraints

• URL: http://arxiv.org/abs/2412.09342v1
• Date: Thu, 12 Dec 2024 15:10:22 GMT
• Title: Diffusion Predictive Control with Constraints
• Authors: Ralf Römer, Alexander von Rohr, Angela P. Schoellig,
• Abstract summary: Diffusion predictive control with constraints (DPCC) An algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data. We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.
• Score: 51.91057765703533
• License:
• Abstract: Diffusion models have recently gained popularity for policy learning in robotics due to their ability to capture high-dimensional and multimodal distributions. However, diffusion policies are inherently stochastic and typically trained offline, limiting their ability to handle unseen and dynamic conditions where novel constraints not represented in the training data must be satisfied. To overcome this limitation, we propose diffusion predictive control with constraints (DPCC), an algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data. DPCC uses constraint tightening and incorporates model-based projections into the denoising process of a trained trajectory diffusion model. This allows us to generate constraint-satisfying, dynamically feasible, and goal-reaching trajectories for predictive control. We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.

Related papers

• Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence [11.400431211239958]
  Diffusion models have emerged as powerful tools for generative modeling. We propose a control framework for fine-tuning diffusion models. We show that PI-FT achieves global convergence at a linear rate.
  arXiv  Detail & Related papers  (2024-12-24T04:55:46Z)
• Efficient Text-driven Motion Generation via Latent Consistency Training [21.348658259929053]
  We propose a motion latent consistency training framework (MLCT) to solve nonlinear reverse diffusion trajectories. By combining these enhancements, we achieve stable and consistency training in non-pixel modality and latent representation spaces.
  arXiv  Detail & Related papers  (2024-05-05T02:11:57Z)
• Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting [11.106812447960186]
  We introduce a novel trajectory generator named Controllable Diffusion Trajectory (CDT) CDT integrates information and social interactions into a Transformer-based conditional denoising diffusion model to guide the prediction of future trajectories. To ensure multimodality, we incorporate behavioral tokens to direct the trajectory's modes, such as going straight, turning right or left.
  arXiv  Detail & Related papers  (2024-02-06T13:16:54Z)
• Compositional Diffusion-Based Continuous Constraint Solvers [98.1702285470628]
  This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP), derives global solutions to CCSPs.
  arXiv  Detail & Related papers  (2023-09-02T15:20:36Z)
• Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning [70.20191211010847]
  Offline reinforcement learning (RL) aims to learn an optimal policy using a previously collected static dataset. We introduce Diffusion Q-learning (Diffusion-QL) that utilizes a conditional diffusion model to represent the policy. We show that our method can achieve state-of-the-art performance on the majority of the D4RL benchmark tasks.
  arXiv  Detail & Related papers  (2022-08-12T09:54:11Z)
• Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Control [64.61499213110334]
  We seek a mechanism to constrain the agent to states and actions that resemble those that it was trained on. In control theory, Lyapunov stability and control-invariant sets allow us to make guarantees about controllers. density models allow us to estimate the training data distribution.
  arXiv  Detail & Related papers  (2022-06-21T16:49:09Z)
• Model-based Safe Reinforcement Learning using Generalized Control Barrier Function [6.556257209888797]
  This paper proposes a model-based feasibility enhancement technique of constrained RL. By using the model information, the policy can be optimized safely without violating actual safety constraints. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.
  arXiv  Detail & Related papers  (2021-03-02T08:17:38Z)
• Chance-Constrained Control with Lexicographic Deep Reinforcement Learning [77.34726150561087]
  This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.
  arXiv  Detail & Related papers  (2020-10-19T13:09:14Z)
• Constrained Model-based Reinforcement Learning with Robust Cross-Entropy Method [30.407700996710023]
  This paper studies the constrained/safe reinforcement learning problem with sparse indicator signals for constraint violations. We employ the neural network ensemble model to estimate the prediction uncertainty and use model predictive control as the basic control framework. The results show that our approach learns to complete the tasks with a much smaller number of constraint violations than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-10-15T18:19:35Z)
• Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions [96.63967125746747]
  Reinforcement learning framework learns the model uncertainty present in the CBF and CLF constraints. RL-CBF-CLF-QP addresses the problem of model uncertainty in the safety constraints.
  arXiv  Detail & Related papers  (2020-04-16T10:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.