Coarse-to-Control: Action-Token Planning for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2606.07107v1
• Date: Fri, 05 Jun 2026 10:01:37 GMT
• Title: Coarse-to-Control: Action-Token Planning for Vision-Language-Action Models
• Abstract summary: Coarse-to-Control is a plan-execute VLA that introduces planning in the action-token space.<n>We show that action-token planning consistently improves over direct action generation.
• Score: 101.44987067974999
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most vision-language-action (VLA) models map observations directly to actions without explicit intermediate planning, which limits performance on long-horizon tasks where early mistakes compound. We propose Coarse-to-Control, a plan-execute VLA that introduces planning natively in the action-token space. The key idea is to let the policy first predict a compact sequence of coarse action tokens that summarize the intended future trajectory, and then generate executable action tokens conditioned on this plan. Because both planning and execution share a unified discrete action vocabulary, the plan stays close to the control manifold and provides directly actionable guidance rather than an abstract hint that must be translated back to motor commands. Experiments on LIBERO, SimplerEnv-WidowX, and real-world manipulation tasks show that action-token planning consistently improves over direct action generation, with the largest gains on long-horizon multi-stage tasks.

Related papers

• WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory [50.19449886428936]
  We introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory.<n>The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning.<n>Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following, and in-context adaptation.
  arXiv  Detail & Related papers  (2026-07-21T08:25:37Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models [14.109112325287208]
  We study the study of latent action supervision from two perspectives: (i) regularizing the trajectory via image-based latent actions, and (ii) unifying the target space with action-based latent actions.<n>Our results reveal a formulation-task correspondence, whereas action-based latent actions excel at complex motor coordination.
  arXiv  Detail & Related papers  (2026-05-06T09:27:07Z)
• Long-Horizon Manipulation via Trace-Conditioned VLA Planning [55.80061850746898]
  LoHo-Manip is a framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM.<n>LoHo-Manip predicts a progress-aware remaining plan that combines a subtask sequence with an explicit done + remaining split as lightweight language memory.<n>The executor VLA is adapted to condition on the rendered trace, thereby turning long-horizon decision-making into repeated local control.
  arXiv  Detail & Related papers  (2026-04-23T17:59:04Z)
• The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas [56.62286434195321]
  This paper systematically studies the effectiveness of two different action representations.<n>We propose cognitive bandwidth perspective as a conceptual framework to qualitatively understand the differences.<n>We provide an actionable guide for building more capable PwS agents for better scalable autonomy.
  arXiv  Detail & Related papers  (2025-10-08T14:47:40Z)
• ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning [47.27336786187929]
  Vision-language-action (VLA) reasoning tasks require agents to interpret multimodal instructions, perform long-horizon planning, and act adaptively in dynamic environments.<n>Existing approaches typically train VLA models in an end-to-end fashion, directly mapping inputs to actions without explicit reasoning.<n>We propose ThinkAct, a dual-system framework that bridges high-level reasoning with low-level action execution via reinforced visual latent planning.
  arXiv  Detail & Related papers  (2025-07-22T17:59:46Z)
• Enhancing Visual Planning with Auxiliary Tasks and Multi-token Prediction [41.63965006043724]
  Visual Planning for Assistance (VPA) aims to predict a sequence of user actions required to achieve a specified goal based on a video showing the user's progress.<n>Recent advances in multimodal large language models (MLLMs) have shown promising results in video understanding.<n>We identify two challenges in training large MLLMs for video-based planning tasks.
  arXiv  Detail & Related papers  (2025-07-20T21:39:05Z)
• Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos [32.71627274876863]
  We address the challenge of procedure planning in instructional videos, aiming to generate coherent and task-aligned action sequences from start and end visual observations.<n>Previous work has mainly relied on text-level supervision to bridge the gap between observed states and unobserved actions, but it struggles with capturing intricate temporal relationships among actions.<n>We propose the Masked Temporal Interpolation Diffusion Diffusion model that introduces a latent space temporal temporal module within the diffusion model.
  arXiv  Detail & Related papers  (2025-07-04T08:54:59Z)
• Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models [63.765846080050906]
  This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location.<n>Experiments show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-12T12:38:20Z)
• Efficient Robotic Policy Learning via Latent Space Backward Planning [17.770562202624962]
  Current robotic planning methods often rely on predicting multi-frame images with full pixel details.<n>We propose a Latent Space Backward Planning scheme (LBP), which begins by grounding the task into final latent goals.<n>We show that LBP outperforms existing fine-grained and forward planning methods, achieving SOTA performance.
  arXiv  Detail & Related papers  (2025-05-11T06:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.