Compositional Visual Planning via Inference-Time Diffusion Scaling

• URL: http://arxiv.org/abs/2603.02646v1
• Date: Tue, 03 Mar 2026 06:24:06 GMT
• Title: Compositional Visual Planning via Inference-Time Diffusion Scaling
• Abstract summary: Diffusion models excel at short-horizon robot planning, yet scaling them to long-horizon tasks remains challenging due to computational constraints and limited training data.<n>We propose that the key to stable compositional generation lies in enforcing boundary agreement on the estimated clean data.<n>Our method formulates long-horizon planning as inference over a chain-structured factor graph of overlapping video chunks, where pretrained short-horizon video diffusion models provide local priors.
• Score: 41.86377754505579
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models excel at short-horizon robot planning, yet scaling them to long-horizon tasks remains challenging due to computational constraints and limited training data. Existing compositional approaches stitch together short segments by separately denoising each component and averaging overlapping regions. However, this suffers from instability as the factorization assumption breaks down in noisy data space, leading to inconsistent global plans. We propose that the key to stable compositional generation lies in enforcing boundary agreement on the estimated clean data (Tweedie estimates) rather than on noisy intermediate states. Our method formulates long-horizon planning as inference over a chain-structured factor graph of overlapping video chunks, where pretrained short-horizon video diffusion models provide local priors. At inference time, we enforce boundary agreement through a novel combination of synchronous and asynchronous message passing that operates on Tweedie estimates, producing globally consistent guidance without requiring additional training. Our training-free framework demonstrates significant improvements over existing baselines, effectively generalizing to unseen start-goal combinations that were not present in the original training data. Project website: https://comp-visual-planning.github.io/

Related papers

• Beyond Binary: Continuous State Optimization with Graph-Structured Objectives [61.561691314533135]
  Large-scale learning systems often face the challenge of balancing multiple, potentially competing objectives.<n>We propose an algorithm that performs lazy updates to minimize switching costs while maintaining near-optimal regret.
  arXiv  Detail & Related papers  (2026-08-10T09:46:28Z)
• Surprise Forcing: What to Remember, When to Skip in Long Video Generation [65.0022589149937]
  Streaming autoregressive diffusion makes minute-scale video synthesis practical, but its bounded context and fixed denoising schedule allocate resources uniformly.<n>We introduce Surprise Forcing, a training-free framework that treats both limitations as online resource-allocation problems.<n> Experiments on VBench, VBench-Long, and VBench-2.0 show that the proposed allocation strategy improves long-horizon consistency and visual quality while retaining real-time streaming throughput.
  arXiv  Detail & Related papers  (2026-07-20T18:37:58Z)
• Plan First, Diffuse Later: Extrinsic Graph Guidance for Long-Horizon Diffusion Planning [44.52185127499181]
  Compositional diffusion models offer a promising route to long-horizon planning.<n>But enforcing local behavior over long chains is often insufficient for a coherent global structure to emerge.<n>Recent works tackle this limitation through intrinsic search, which explores multiple paths during the denoising process.
  arXiv  Detail & Related papers  (2026-05-16T07:53:04Z)
• Refining Compositional Diffusion for Reliable Long-Horizon Planning [31.40044829548484]
  We propose a training-free guidance method that steers compositional sampling toward high-density, globally coherent plans.<n> Experiments on challenging long-horizon tasks from OGBench, including locomotion, object manipulation, and pixel-based observations, demonstrate that RCD consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-05-04T18:44:19Z)
• Amortizing Trajectory Diffusion with Keyed Drift Fields [2.5352713493505785]
  Keyed Drifting Policies is a one-step trajectory generator trained with a drift-field objective.<n>It repels generated trajectories toward condition-matched dataset windows.<n>It achieves strong performance with one-step inference and substantially lower planning latency than diffusion sampling.
  arXiv  Detail & Related papers  (2026-03-14T17:56:15Z)
• UnLoc: Leveraging Depth Uncertainties for Floorplan Localization [80.55849461031879]
  UnLoc is an efficient data-driven solution for sequential camera localization within floorplans.<n>We introduce a novel probabilistic model that incorporates uncertainty estimation, modeling depth predictions as explicit probability distributions.<n>We evaluate UnLoc on large-scale synthetic and real-world datasets, demonstrating significant improvements in terms of accuracy and robustness.
  arXiv  Detail & Related papers  (2025-09-14T14:45:43Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Generative Trajectory Stitching through Diffusion Composition [29.997765496994457]
  CompDiffuser is a novel generative approach that can solve new tasks by learning to compositionally stitch together shorter trajectory chunks from previously seen tasks.<n>We conduct experiments on benchmark tasks of various difficulties, covering different environment sizes, agent state dimension, trajectory types, training data quality, and show that CompDiffuser significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2025-03-07T05:22:52Z)
• FLARE: A New Federated Learning Framework with Adjustable Learning Rates over Resource-Constrained Wireless Networks [20.048146776405005]
  Wireless federated learning (WFL) suffers from heterogeneity prevailing in the data distributions, computing powers, and channel conditions. This paper presents a new idea with Federated Learning Adjusted leaning ratE (FLR ratE) Experiments that FLARE consistently outperforms the baselines.
  arXiv  Detail & Related papers  (2024-04-23T07:48:17Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• Compositional Diffusion-Based Continuous Constraint Solvers [98.1702285470628]
  This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP), derives global solutions to CCSPs.
  arXiv  Detail & Related papers  (2023-09-02T15:20:36Z)
• $\texttt{FedBC}$: Calibrating Global and Local Models via Federated Learning Beyond Consensus [66.62731854746856]
  In federated learning (FL), the objective of collaboratively learning a global model through aggregation of model updates across devices tends to oppose the goal of personalization via local information. In this work, we calibrate this tradeoff in a quantitative manner through a multi-criterion-based optimization. We demonstrate that $texttFedBC$ balances the global and local model test accuracy metrics across a suite datasets.
  arXiv  Detail & Related papers  (2022-06-22T02:42:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.