Warm-Starting Collision-Free Model Predictive Control With Object-Centric Diffusion

• URL: http://arxiv.org/abs/2601.02873v2
• Date: Wed, 21 Jan 2026 09:53:52 GMT
• Title: Warm-Starting Collision-Free Model Predictive Control With Object-Centric Diffusion
• Abstract summary: We show that combining diffusion-based warm-starting conditioned with a latent object-centric representation of the scene yields reliable and efficient motion generation under strict time limits.<n>Our approach conditions a diffusion transformer on the system state, task, and surroundings, using an object-centric slot attention mechanism.<n>The sampled trajectories are refined by an optimal control problem that enforces rigid-body dynamics and signed-distance collision constraints.
• Score: 38.363402738466334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acting in cluttered environments requires predicting and avoiding collisions while still achieving precise control. Conventional optimization-based controllers can enforce physical constraints, but they struggle to produce feasible solutions quickly when many obstacles are present. Diffusion models can generate diverse trajectories around obstacles, yet prior approaches lacked a general and efficient way to condition them on scene structure. In this paper, we show that combining diffusion-based warm-starting conditioned with a latent object-centric representation of the scene and with a collision-aware model predictive controller (MPC) yields reliable and efficient motion generation under strict time limits. Our approach conditions a diffusion transformer on the system state, task, and surroundings, using an object-centric slot attention mechanism to provide a compact obstacle representation suitable for control. The sampled trajectories are refined by an optimal control problem that enforces rigid-body dynamics and signed-distance collision constraints, producing feasible motions in real time. On benchmark tasks, this hybrid method achieved markedly higher success rates and lower latency than sampling-based planners or either component alone. Real-robot experiments with a torque-controlled Panda confirm reliable and safe execution with MPC.

Related papers

• Predictive Relative-Velocity Steering for Safe Robotic Manipulator Teleoperation in Dynamic Environments [7.297209090323272]
  We propose a lightweight and modular framework for proactive collision avoidance.<n>The framework first predicts potential collisions based on time-to-collision with integrated overshoot protection.<n>The deflection changes only the direction of the relative velocity while preserving its magnitude.
  arXiv  Detail & Related papers  (2026-08-13T14:17:12Z)
• A Task-Space Receding Horizon Controller for Fast Collision Avoidance [0.08594140167290099]
  Real-time collision avoidance for robotic manipulators requires fast reactions to unexpected obstacle motion.<n>Full model predictive control can provide this foresight, but its online cost may grow quickly with horizon length, model fidelity, and the number of active geometric constraints.<n>We present a task-space receding-horizon controller that uses a short contact-consistent rollout to generate a terminal kinematic reference.
  arXiv  Detail & Related papers  (2026-07-17T08:13:12Z)
• Model-Based Diffusion Optimal Control for Multi-Robot Motion Planning [5.5818653283217055]
  Multi-Robot Motion Planning in continuous environments is challenging due to the growth of the joint trajectory space and the difficulty of enforcing dynamic feasibility and hard safety constraints.<n>Recent approaches recast trajectory planning as probabilistic inference, sampling from a posterior over trajectories using diffusion models whose score functions are learned from demonstration data.<n>We introduce Model-Based Diffusion Optimal Control (MDOC), a model-based diffusion planner that efficiently produces dynamically feasible trajectories without relying on data.
  arXiv  Detail & Related papers  (2026-07-14T06:54:03Z)
• Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control [41.782471938706344]
  For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible.<n>We propose an inference-time optimization framework that couples the behavior generation to physical feasibility.<n>We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints.
  arXiv  Detail & Related papers  (2026-06-23T06:49:30Z)
• SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors [48.60897189520386]
  We introduce the JEPA-style model for real-time quadrotor control.<n>We combine a latent dynamics model with a novel physics-inspired prober that maps frozen latents to interpretable state.<n>To reduce the dependence on expensive and unsafe real-world data collection, we develop a structured pipeline for automated dataset generation.
  arXiv  Detail & Related papers  (2026-06-22T15:00:59Z)
• Imitation learning-based spacecraft rendezvous and docking method with Expert Demonstration [8.727025636218528]
  Existing spacecraft rendezvous and docking control methods rely on predefined dynamic models.<n>This paper proposes an Imitation Learning-based framework that learns control policies from expert demonstrations.<n>The proposed framework achieves accurate and energy-efficient model-free rendezvous and docking control.
  arXiv  Detail & Related papers  (2026-01-19T10:58:26Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Multi-Agent Path Finding in Continuous Spaces with Projected Diffusion Models [57.45019514036948]
  Multi-Agent Path Finding (MAPF) is a fundamental problem in robotics.<n>This work proposes a novel approach that integrates constrained optimization with diffusion models for MAPF in continuous spaces.
  arXiv  Detail & Related papers  (2024-12-23T21:27:19Z)
• Diffusion Predictive Control with Constraints [51.91057765703533]
  Diffusion predictive control with constraints (DPCC) is an algorithm for diffusion-based control with explicit state and action constraints.<n>We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints.
  arXiv  Detail & Related papers  (2024-12-12T15:10:22Z)
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [56.66677293607114]
  We propose Code-as-Monitor (CaM) for both open-set reactive and proactive failure detection.<n>To enhance the accuracy and efficiency of monitoring, we introduce constraint elements that abstract constraint-related entities.<n>Experiments show that CaM achieves a 28.7% higher success rate and reduces execution time by 31.8% under severe disturbances.
  arXiv  Detail & Related papers  (2024-12-05T18:58:27Z)
• Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
  In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability. We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
  arXiv  Detail & Related papers  (2024-02-04T15:54:03Z)
• Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planning [3.074694788117593]
  Diffusion models have risen as a powerful tool in robotics due to their flexibility and multi-modality. We present a method that simultaneously generates only reachable goals and plans motions that avoid obstacles.
  arXiv  Detail & Related papers  (2023-10-19T09:39:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.