Diffusion Predictive Control with Constraints

• URL: http://arxiv.org/abs/2412.09342v1
• Date: Thu, 12 Dec 2024 15:10:22 GMT
• Title: Diffusion Predictive Control with Constraints
• Authors: Ralf Römer, Alexander von Rohr, Angela P. Schoellig,
• Abstract summary: Diffusion predictive control with constraints (DPCC)<n>An algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data.<n>We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.
• Score: 51.91057765703533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have recently gained popularity for policy learning in robotics due to their ability to capture high-dimensional and multimodal distributions. However, diffusion policies are inherently stochastic and typically trained offline, limiting their ability to handle unseen and dynamic conditions where novel constraints not represented in the training data must be satisfied. To overcome this limitation, we propose diffusion predictive control with constraints (DPCC), an algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data. DPCC uses constraint tightening and incorporates model-based projections into the denoising process of a trained trajectory diffusion model. This allows us to generate constraint-satisfying, dynamically feasible, and goal-reaching trajectories for predictive control. We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.

Related papers

• Aligning Diffusion Model with Problem Constraints for Trajectory Optimization [0.6629765271909505]
  We propose a novel approach that aligns diffusion models explicitly with problem-specific constraints. Our approach is well-suited for integration into the Dynamic Data-driven Application Systems (DDDAS) framework.
  arXiv  Detail & Related papers  (2025-04-01T01:46:05Z)
• Constrained Language Generation with Discrete Diffusion Models [61.81569616239755]
  We present Constrained Discrete Diffusion (CDD), a novel method for enforcing constraints on natural language by integrating discrete diffusion models with differentiable optimization. We show how this technique can be applied to satisfy a variety of natural language constraints, including (i) toxicity mitigation by preventing harmful content from emerging, (ii) character and sequence level lexical constraints, and (iii) novel molecule sequence generation with specific property adherence.
  arXiv  Detail & Related papers  (2025-03-12T19:48:12Z)
• Estimating Control Barriers from Offline Data [14.241303913878887]
  We propose a novel framework for learning neural CBFs through a fixed, sparsely-labeled dataset collected prior to training. With limited amount of offline data, it achieves state-of-the-art performance for dynamic obstacle avoidance.
  arXiv  Detail & Related papers  (2025-02-21T04:55:20Z)
• Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learning [13.332006760984122]
  Action-Constrained Reinforcement Learning (ACRL) employs a projection layer after the policy network to correct the action. Recent methods were proposed to train generative models to learn a differentiable mapping between latent variables and feasible actions.
  arXiv  Detail & Related papers  (2025-02-08T12:58:26Z)
• Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence [11.400431211239958]
  Diffusion models have emerged as powerful tools for generative modeling. We propose a control framework for fine-tuning diffusion models. We show that PI-FT achieves global convergence at a linear rate.
  arXiv  Detail & Related papers  (2024-12-24T04:55:46Z)
• Adding Conditional Control to Diffusion Models with Reinforcement Learning [59.295203871547336]
  Diffusion models are powerful generative models that allow for precise control over the characteristics of the generated samples. This work presents a novel method based on reinforcement learning (RL) to add additional controls, leveraging an offline dataset.
  arXiv  Detail & Related papers  (2024-06-17T22:00:26Z)
• Efficient Text-driven Motion Generation via Latent Consistency Training [21.348658259929053]
  We propose a motion latent consistency training framework (MLCT) to solve nonlinear reverse diffusion trajectories.<n>By combining these enhancements, we achieve stable and consistency training in non-pixel modality and latent representation spaces.
  arXiv  Detail & Related papers  (2024-05-05T02:11:57Z)
• Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting [11.106812447960186]
  We introduce a novel trajectory generator named Controllable Diffusion Trajectory (CDT) CDT integrates information and social interactions into a Transformer-based conditional denoising diffusion model to guide the prediction of future trajectories. To ensure multimodality, we incorporate behavioral tokens to direct the trajectory's modes, such as going straight, turning right or left.
  arXiv  Detail & Related papers  (2024-02-06T13:16:54Z)
• Compositional Diffusion-Based Continuous Constraint Solvers [98.1702285470628]
  This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP), derives global solutions to CCSPs.
  arXiv  Detail & Related papers  (2023-09-02T15:20:36Z)
• Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning [70.20191211010847]
  Offline reinforcement learning (RL) aims to learn an optimal policy using a previously collected static dataset. We introduce Diffusion Q-learning (Diffusion-QL) that utilizes a conditional diffusion model to represent the policy. We show that our method can achieve state-of-the-art performance on the majority of the D4RL benchmark tasks.
  arXiv  Detail & Related papers  (2022-08-12T09:54:11Z)
• Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Control [64.61499213110334]
  We seek a mechanism to constrain the agent to states and actions that resemble those that it was trained on. In control theory, Lyapunov stability and control-invariant sets allow us to make guarantees about controllers. density models allow us to estimate the training data distribution.
  arXiv  Detail & Related papers  (2022-06-21T16:49:09Z)
• Adjacency constraint for efficient hierarchical reinforcement learning [25.15808501708926]
  Goal-conditioned Reinforcement Learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. HRL often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is large. We show that this problem can be effectively alleviated by restricting the high-level action space to a $k$-step adjacent region of the current state.
  arXiv  Detail & Related papers  (2021-10-30T09:26:45Z)
• Model-based Safe Reinforcement Learning using Generalized Control Barrier Function [6.556257209888797]
  This paper proposes a model-based feasibility enhancement technique of constrained RL. By using the model information, the policy can be optimized safely without violating actual safety constraints. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.
  arXiv  Detail & Related papers  (2021-03-02T08:17:38Z)
• Constrained Model-based Reinforcement Learning with Robust Cross-Entropy Method [30.407700996710023]
  This paper studies the constrained/safe reinforcement learning problem with sparse indicator signals for constraint violations. We employ the neural network ensemble model to estimate the prediction uncertainty and use model predictive control as the basic control framework. The results show that our approach learns to complete the tasks with a much smaller number of constraint violations than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-10-15T18:19:35Z)
• Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions [96.63967125746747]
  Reinforcement learning framework learns the model uncertainty present in the CBF and CLF constraints. RL-CBF-CLF-QP addresses the problem of model uncertainty in the safety constraints.
  arXiv  Detail & Related papers  (2020-04-16T10:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.