LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation

• URL: http://arxiv.org/abs/2608.03701v1
• Date: Tue, 04 Aug 2026 14:06:15 GMT
• Title: LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
• Abstract summary: LiLa-WAM is a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU.<n>For task specification, we propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space.
• Score: 26.29942570664068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World-action modeling has emerged as a promising paradigm for robotic control, as it empowers models to go beyond reacting to observations and anticipate how a scene will evolve. However, existing WAMs often incur substantial computational overhead. Pixel-space methods often allocate substantial capacity to visual details that may not be directly relevant to control, while some latent-space methods require multi-stage training to construct the reasoning space. The resulting training cost can make such methods difficult to train under modest computational budgets. In this work, we propose LiLa-WAM, a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU. Its core design is a compact latent reasoning space jointly shaped by future-state prediction and action generation, which keeps the model lightweight while remaining well aligned with control. For task specification, we further propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space. Experiments on RoboTwin~2.0, LIBERO, and real-robot tasks demonstrate LiLa-WAM's effectiveness, achieving 90.48\% success across 50 RoboTwin tasks with single-GPU training.

Related papers

• Guava: An Effective and Universal Harness for Embodied Manipulation [74.34187069605844]
  We develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model.<n>Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation.
  arXiv  Detail & Related papers  (2026-06-16T18:09:26Z)
• LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies [31.88655699486955]
  We present LaWAM, a latent-action-conditioned Latent World Model (LaWM)<n>LaWAM exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video.<n>LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
  arXiv  Detail & Related papers  (2026-06-14T12:06:58Z)
• Light-WAM: Efficient World Action Models with State-Fusion Action Decoding [15.384126562001027]
  Light-WAM is a lightweight World Action Model for efficient robot manipulation.<n>It is built with a compact video backbone and performs future-video supervision in a downsampled latent space.<n>Experiments demonstrate that Light-WAM maintains strong performance on LIBERO and achieves usable multi-task performance on RoboTwin 2.0.
  arXiv  Detail & Related papers  (2026-06-06T15:58:12Z)
• Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments [96.23886784364997]
  Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation.<n>We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack to continuous action and trajectory generation.
  arXiv  Detail & Related papers  (2026-05-28T17:36:31Z)
• Large Video Planner Enables Generalizable Robot Control [117.49024534548319]
  General-purpose robots require decision-making models that generalize across diverse tasks and environments.<n>Recent works build robot foundation models by extending multimodal large language models (LMs) with action outputs, creating vision--action (VLA) systems.<n>We explore an alternative paradigm of using large-scale video pretraining as a primary modality for building robot foundation models.
  arXiv  Detail & Related papers  (2025-12-17T18:35:54Z)
• Affordance-Guided Reinforcement Learning via Visual Prompting [51.361977466993345]
  Keypoint-based Affordance Guidance for Improvements (KAGI) is a method leveraging rewards shaped by vision-language models (VLMs) for autonomous RL.<n>On diverse real-world manipulation tasks specified by natural language descriptions, KAGI improves the sample efficiency of autonomous RL and enables successful task completion in 30K online fine-tuning steps.
  arXiv  Detail & Related papers  (2024-07-14T21:41:29Z)
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  We introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations.<n>First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets.<n>We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation [39.44358155600282]
  We introduce RoboMamba, an end-to-end robotic VLA model that delivers both robotic reasoning and action capabilities.<n>Specifically, we first integrate the vision encoder with Mamba, aligning visual tokens with language embedding through co-training.<n>We find that once RoboMamba possesses sufficient reasoning capability, it can acquire manipulation skills with minimal fine-tuning parameters.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• Bridging Language, Vision and Action: Multimodal VAEs in Robotic Manipulation Tasks [0.0]
  In this work, we focus on unsupervised vision-language--action mapping in the area of robotic manipulation.<n>We propose a model-invariant training alternative that improves the models' performance in a simulator by up to 55%.<n>Our work thus also sheds light on the potential benefits and limitations of using the current multimodal VAEs for unsupervised learning of robotic motion trajectories.
  arXiv  Detail & Related papers  (2024-04-02T13:25:16Z)
• VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models [38.503337052122234]
  Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation. We aim to synthesize robot trajectories for a variety of manipulation tasks given an open-set of instructions and an open-set of objects. We demonstrate how the proposed framework can benefit from online experiences by efficiently learning a dynamics model for scenes that involve contact-rich interactions.
  arXiv  Detail & Related papers  (2023-07-12T07:40:48Z)
• AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation [50.737355245505334]
  We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks. The resulting dataset AlphaBlock consists of 35 comprehensive high-level tasks of multi-step text plans and paired observation.
  arXiv  Detail & Related papers  (2023-05-30T09:54:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.