Latent Diffusion Policy: Shaping Latent Spaces for Diffusion-Based Robotic Manipulation

• URL: http://arxiv.org/abs/2606.08657v1
• Date: Sun, 07 Jun 2026 14:53:28 GMT
• Title: Latent Diffusion Policy: Shaping Latent Spaces for Diffusion-Based Robotic Manipulation
• Abstract summary: We introduce Latent Diffusion Policy (LDP), a framework performing flow matching in a deliberately shaped latent space.<n>By absorbing scene understanding into an observation-conditioned CVAE encoder, LDP concentrates the conditional distribution of each observation.<n>On coordination-intensive tasks from RoboTwin 2.0, LDP outperforms DP3 by a substantial margin and transfers effectively to real-world bimanual deployments.
• Score: 2.606982259886594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based visuomotor policies operating directly in raw action spaces conflate scene comprehension with trajectory generation within a single denoising process. The resulting velocity field must simultaneously encode scene information and generate precise trajectories, increasing learning complexity and limiting performance on tasks demanding precise temporal coordination across multiple arms. To simplify this joint learning problem, we introduce Latent Diffusion Policy (LDP), a two-stage framework performing flow matching in a deliberately shaped latent space. By absorbing scene understanding into an observation-conditioned CVAE encoder, LDP concentrates the conditional distribution of each observation. Consequently, the flow model avoids implicitly resolving scene-dependent structures; instead, it generates within a pre-concentrated distribution featuring a smoother velocity field, simplifying learning from limited demonstrations. Furthermore, to capture temporal dependencies among latent tokens, LDP trains with per-token diffusion forcing and employs staircase inference sampling to resolve the resulting distributional mismatch. We also propose reconstruction FID (rFID) as a lightweight proxy predicting downstream task success solely from latent space statistics. On coordination-intensive tasks from RoboTwin 2.0, LDP outperforms DP3 by a substantial margin and transfers effectively to real-world bimanual deployments.

Related papers

• Motion Planning with Model-Based Diffusion via Constraint Optimization and Adaptive Scheduling [7.761673975696802]
  Single-Robot Motion Planning (SRMP) is challenging under complex constraints and computational limits.<n>Recent Model-Based Diffusion (MBD) recasts the SRMP to guide samples from a posterior diffusion trajectories.<n>Our method achieves higher safety & success rates, faster convergence, and lower final costs than baseline planners.
  arXiv  Detail & Related papers  (2026-07-16T01:04:32Z)
• SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation [65.81476183180527]
  We introduce SegDiff, a closed-loop visuomotor policy that integrates the strengths of both paradigms.<n>SegDiff decomposes demonstrations into motion segments between keyposes and learns to predict the continuous trajectory from the current state to the next keypose.<n>SegDiff demonstrates significant performance gains over existing approaches across various simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2026-07-13T02:48:03Z)
• GLIDE: Graph-guided Leap Inference for Diffusion Estimation of Spatio-Temporal Point Processes [19.756452407351986]
  We propose a conditional diffusion framework for temporal-temporal point processes (STPPs)<n>GLIDE organizes historical events into a multi-scale historical graph and encodes spatial topology through a dual-stream architecture.<n> Experiments on multiple real-world datasets show that GLIDE improves both distribution fitting and next-event prediction, with the largest gains appearing on the spatial side.
  arXiv  Detail & Related papers  (2026-05-31T14:56:24Z)
• Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration [3.0222726254970174]
  Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning.<n>This paper presents a hybrid belief-reinforcement learning framework to address this gap.<n>Results show 10.8% higher cumulative reward and 38% faster convergence over baselines.
  arXiv  Detail & Related papers  (2026-03-04T00:00:34Z)
• SENTINEL: Stagewise Integrity Verification for Pipeline Parallel Decentralized Training [54.8494905524997]
  Decentralized training introduces critical security risks when executed across untrusted, geographically distributed nodes.<n>We propose SENTINEL, a verification mechanism for pipeline parallelism (PP) training without duplication.<n>Experiments demonstrate successful training of up to 4B- parameter LLMs across untrusted distributed environments with up to 176 workers while maintaining model convergence and performance.
  arXiv  Detail & Related papers  (2026-03-03T23:51:10Z)
• LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization [15.308350522323588]
  We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide.<n>On several popular and challenging datasets, LEAR achieves superior performance over the best prior method.
  arXiv  Detail & Related papers  (2026-03-02T13:18:25Z)
• Causal Autoregressive Diffusion Language Model [70.7353007255797]
  CARD reformulates the diffusion process within a strictly causal attention mask, enabling dense, per-token supervision in a single forward pass.<n>Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation.
  arXiv  Detail & Related papers  (2026-01-29T17:38:29Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• CAPE: Context-Aware Diffusion Policy Via Proximal Mode Expansion for Collision Avoidance [15.311155448797386]
  contexts-aware diffusion policy via Proximal mode Expansion (CAPE)<n>CAPE expands trajectory distribution modes with context-aware prior and guidance at inference.<n>We evaluate CAPE on diverse manipulation tasks in cluttered unseen simulated and real-world settings.
  arXiv  Detail & Related papers  (2025-11-27T21:53:09Z)
• Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model [62.889356203346985]
  We propose DUal-STream diffusion (DUST), a world-model augmented VLA framework that handles the modality conflict.<n>DUST achieves up to 6% gains over a standard VLA baseline and implicit world-modeling methods.<n>On real-world tasks with the Franka Research 3, DUST outperforms baselines in success rate by 13%.
  arXiv  Detail & Related papers  (2025-10-31T16:32:12Z)
• Projected Coupled Diffusion for Test-Time Constrained Joint Generation [49.69610867216755]
  We propose Projected Coupled Diffusion (PCD), a novel test-time framework for constrained joint generation.<n>PCD introduces a coupled guidance term into the generative dynamics to encourage coordination between diffusion models and incorporates a projection step at each diffusion step to enforce hard constraints.<n>Our results show improved coupling effects and guaranteed constraint satisfaction without incurring excessive computational costs.
  arXiv  Detail & Related papers  (2025-08-14T11:05:31Z)
• Temporal Context Aggregation Network for Temporal Action Proposal Refinement [93.03730692520999]
  Temporal action proposal generation is a challenging yet important task in the video understanding field. Current methods still suffer from inaccurate temporal boundaries and inferior confidence used for retrieval. We propose TCANet to generate high-quality action proposals through "local and global" temporal context aggregation.
  arXiv  Detail & Related papers  (2021-03-24T12:34:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.