Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation

• URL: http://arxiv.org/abs/2603.12553v1
• Date: Fri, 13 Mar 2026 01:33:48 GMT
• Title: Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation
• Abstract summary: We propose StructVLA, which reformulates a generative world model into an explicit structured planner for reliable control.<n>We implement this approach through a two-stage training paradigm with a unified discrete token vocabulary.<n>In our experiments, StructVLA achieves strong average success rates of 75.0% on SimplerEnv-WidowX and 94.8% on LIBERO.
• Score: 43.5447478385855
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent world-model-based Vision-Language-Action (VLA) architectures have improved robotic manipulation through predictive visual foresight. However, dense future prediction introduces visual redundancy and accumulates errors, causing long-horizon plan drift. Meanwhile, recent sparse methods typically represent visual foresight using high-level semantic subtasks or implicit latent states. These representations often lack explicit kinematic grounding, weakening the alignment between planning and low-level execution. To address this, we propose StructVLA, which reformulates a generative world model into an explicit structured planner for reliable control. Instead of dense rollouts or semantic goals, StructVLA predicts sparse, physically meaningful structured frames. Derived from intrinsic kinematic cues (e.g., gripper transitions and kinematic turning points), these frames capture spatiotemporal milestones closely aligned with task progress. We implement this approach through a two-stage training paradigm with a unified discrete token vocabulary: the world model is first trained to predict structured frames and subsequently optimized to map the structured foresight into low-level actions. This approach provides clear physical guidance and bridges visual planning and motion control. In our experiments, StructVLA achieves strong average success rates of 75.0% on SimplerEnv-WidowX and 94.8% on LIBERO. Real-world deployments further demonstrate reliable task completion and robust generalization across both basic pick-and-place and complex long-horizon tasks.

Related papers

• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System [48.02804864175127]
  HiVLA is a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control.<n>Experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines.
  arXiv  Detail & Related papers  (2026-04-15T17:50:07Z)
• StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models [98.72926158261937]
  We propose a training-free token pruning framework for Visual AutoRegressive models.<n>We employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information.<n>To maintain valid next-scale prediction under sparse tokens, we introduce a nearest neighbor feature propagation strategy.
  arXiv  Detail & Related papers  (2026-03-02T11:35:05Z)
• Scaling World Model for Hierarchical Manipulation Policies [61.736772957803026]
  Vision-Language-Action (VLA) models are promising for generalist robot manipulation but brittle in out-of-distribution settings.<n>We introduce a hierarchical Vision-Language-Action framework our that leverages the generalization of large-scale pre-trained world model.<n>We validate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-11T16:12:33Z)
• SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model [27.54931639768958]
  This paper introduces a novel architecture for trajectory-conditioned forecasting of future 3D scene occupancy.<n>Inspired by attention-based transformer architectures in foundational vision and language models such as GPT and VGGT, we employ a sparse occupancy representation that bypasses the intermediate bird's eye view (BEV) projection and its explicit geometric priors.<n>By avoiding both the finite-capacity constraint of discrete tokenization and the structural limitations of BEV representations, our method achieves state-of-the-art performance on the nuScenes benchmark for 1-3 second occupancy forecasting.
  arXiv  Detail & Related papers  (2025-11-27T02:48:45Z)
• Reinforced Reasoning for Embodied Planning [18.40186665383579]
  Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and natural language goals.<n>We introduce a reinforcement fine-tuning framework that brings R1-style reasoning enhancement into embodied planning.
  arXiv  Detail & Related papers  (2025-05-28T07:21:37Z)
• Human locomotor control timescales depend on the environmental context and sensory input modality [37.48294298569551]
  We present a unified data-driven framework to quantify the control timescales.<n>We apply this framework across tasks including walking and running.<n>Our framework reveals the factors that influence locomotor foot placement control timescales.
  arXiv  Detail & Related papers  (2025-03-20T16:57:15Z)
• Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features [59.892436892964376]
  We investigate the minimal data requirements and architectural adaptations necessary to achieve robust closed-loop performance with vision-based control policies.<n>Our findings are synthesized in Flex (Fly lexically), a framework that uses pre-trained Vision Language Models (VLMs) as frozen patch-wise feature extractors.<n>We demonstrate the effectiveness of this approach on a quadrotor fly-to-target task, where agents trained via behavior cloning successfully generalize to real-world scenes.
  arXiv  Detail & Related papers  (2024-10-16T19:59:31Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.