Multi-Robot Motion Planning from Vision and Language using Heat-Inspired Diffusion

• URL: http://arxiv.org/abs/2512.13090v1
• Date: Mon, 15 Dec 2025 08:43:13 GMT
• Title: Multi-Robot Motion Planning from Vision and Language using Heat-Inspired Diffusion
• Abstract summary: Language-Conditioned Heat-Inspired Diffusion (LCHD) is an end-to-end vision-based framework that generates language-conditioned, collision-free trajectories.<n> LCHD integrates CLIP-based semantic priors with a collision-avoiding diffusion kernel serving as a physical inductive bias.<n> LCHD consistently outperforms prior diffusion-based planners in success rate, while reducing planning latency.
• Score: 6.286759951136632
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion models have recently emerged as powerful tools for robot motion planning by capturing the multi-modal distribution of feasible trajectories. However, their extension to multi-robot settings with flexible, language-conditioned task specifications remains limited. Furthermore, current diffusion-based approaches incur high computational cost during inference and struggle with generalization because they require explicit construction of environment representations and lack mechanisms for reasoning about geometric reachability. To address these limitations, we present Language-Conditioned Heat-Inspired Diffusion (LCHD), an end-to-end vision-based framework that generates language-conditioned, collision-free trajectories. LCHD integrates CLIP-based semantic priors with a collision-avoiding diffusion kernel serving as a physical inductive bias that enables the planner to interpret language commands strictly within the reachable workspace. This naturally handles out-of-distribution scenarios -- in terms of reachability -- by guiding robots toward accessible alternatives that match the semantic intent, while eliminating the need for explicit obstacle information at inference time. Extensive evaluations on diverse real-world-inspired maps, along with real-robot experiments, show that LCHD consistently outperforms prior diffusion-based planners in success rate, while reducing planning latency.

Related papers

• Lagrange: An Open-Vocabulary, Energy-Based Sparse Framework for Generalized End-to-End Driving [12.828441606176447]
  Scaling end-to-end autonomous driving to complex, open-world environments requires perceptual models that generalize to anomalous scenarios and planners that produce kinematically valid trajectories.<n>We propose Lagrange, an open-vocabulary, computationally sparse driving framework based on Masked Latent Fields (MLF)<n>We introduce an intent-driven masked cross-attention module that temporally filters irrelevant entities, decoding the attended tokens into an implicit continuous energy field defined over spatial coordinates.
  arXiv  Detail & Related papers  (2026-06-18T14:18:01Z)
• KnowDiffuser: A Knowledge-Guided Diffusion Planner with LM Reasoning and Prior-Informed Trajectory Initialization [8.04458701181863]
  KnowDiffuser is a knowledge-guided motion planning framework.<n>It integrates semantic understanding of language models with the generative power of diffusion models.<n>It significantly outperforms existing planners in both open-loop and closed-loop evaluations.
  arXiv  Detail & Related papers  (2026-03-11T05:45:29Z)
• Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion [23.834662472392694]
  Masked Vision-Language-Action Diffusion for Autonomous Driving (MVLAD-AD) is a novel framework designed to bridge the gap between efficient planning and semantic explainability.<n>We introduce a discrete action tokenization strategy that constructs a compact codebook of kinematically feasible waypoints from real-world driving distributions.<n>Experiments on nuScenes and derived benchmarks demonstrate that MVLAD-AD achieves superior efficiency and outperforms state-of-the-art autoregressive and diffusion baselines in planning precision.
  arXiv  Detail & Related papers  (2026-02-24T05:59:10Z)
• LLM-Grounded Dynamic Task Planning with Hierarchical Temporal Logic for Human-Aware Multi-Robot Collaboration [17.886091169216538]
  Large Language Models (LLM) enable non-experts to specify openworld multi-robot tasks.<n>LLM plans often lack feasibility and are not efficient, especially in long-horizon scenarios.<n>We propose a neuro-symbolic framework that grounds reasoning into hierarchical specifications.
  arXiv  Detail & Related papers  (2026-02-10T07:11:36Z)
• LaST$_{0}$: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Model [102.60980325911106]
  We propose a framework that enables efficient reasoning before acting through a Latent Spatio-Temporal Chain-of-Thought (CoT)<n>Specifically, we introduce a token-efficient latent CoT space that models future visual dynamics, 3D structural information, and robot proprioceptive states, and further extends these representations across time to enable temporally consistent implicit reasoning trajectories.
  arXiv  Detail & Related papers  (2026-01-08T18:59:53Z)
• Stable Language Guidance for Vision-Language-Action Models [62.80963701282789]
  Residual Semantic Steering is a probabilistic framework that disentangles physical affordance from semantic execution.<n> RSS achieves state-of-the-art robustness, maintaining performance even under adversarial linguistic perturbations.
  arXiv  Detail & Related papers  (2026-01-07T16:16:10Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Grounding Language Models with Semantic Digital Twins for Robotic Planning [6.474368392218828]
  We introduce a novel framework that integrates Semantic Digital Twins (SDTs) with Large Language Models (LLMs)<n>The proposed framework effectively combines high-level reasoning with semantic environment understanding, achieving reliable task completion in the face of uncertainty and failure.
  arXiv  Detail & Related papers  (2025-06-19T17:38:00Z)
• Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space [74.12387631212609]
  We introduce SLED, an alternative approach to speech language modeling by encoding speech waveforms into sequences of continuous latent representations.<n>SLED avoids discretization errors and eliminates the need for the complicated hierarchical architectures common in existing speech language models.<n> Empirical results demonstrate that SLED achieves strong performance in both zero-shot and streaming speech synthesis.
  arXiv  Detail & Related papers  (2025-05-19T14:38:59Z)
• Latent Diffusion Planning for Imitation Learning [78.56207566743154]
  Latent Diffusion Planning (LDP) is a modular approach consisting of a planner and inverse dynamics model.<n>By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data.<n>On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches.
  arXiv  Detail & Related papers  (2025-04-23T17:53:34Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• Simultaneous Multi-Robot Motion Planning with Projected Diffusion Models [57.45019514036948]
  Simultaneous MRMP Diffusion (SMD) is a novel approach integrating constrained optimization into the diffusion sampling process to produce collision-free, kinematically feasible trajectories.<n>The paper introduces a comprehensive MRMP benchmark to evaluate trajectory planning algorithms across scenarios with varying robot densities, obstacle complexities, and motion constraints.
  arXiv  Detail & Related papers  (2025-02-05T20:51:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.