MetaWorld: Skill Transfer and Composition in a Hierarchical World Model for Grounding High-Level Instructions

• URL: http://arxiv.org/abs/2601.17507v1
• Date: Sat, 24 Jan 2026 16:11:45 GMT
• Title: MetaWorld: Skill Transfer and Composition in a Hierarchical World Model for Grounding High-Level Instructions
• Abstract summary: We propose MetaWorld, a hierarchical world model that integrates semantic planning and physical control.<n>We show it outperforms world model-based RL in task completion and motion coherence.
• Score: 2.6994954303270564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid robot loco-manipulation remains constrained by the semantic-physical gap. Current methods face three limitations: Low sample efficiency in reinforcement learning, poor generalization in imitation learning, and physical inconsistency in VLMs. We propose MetaWorld, a hierarchical world model that integrates semantic planning and physical control via expert policy transfer. The framework decouples tasks into a VLM-driven semantic layer and a latent dynamics model operating in a compact state space. Our dynamic expert selection and motion prior fusion mechanism leverages a pre-trained multi-expert policy library as transferable knowledge, enabling efficient online adaptation via a two-stage framework. VLMs serve as semantic interfaces, mapping instructions to executable skills and bypassing symbol grounding. Experiments on Humanoid-Bench show MetaWorld outperforms world model-based RL in task completion and motion coherence. Our code will be found at https://anonymous.4open.science/r/metaworld-2BF4/

Related papers

• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• $M^2$-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills [25.17452377052361]
  Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning.<n>We propose $M2$-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly.<n>To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features.
  arXiv  Detail & Related papers  (2026-04-27T08:44:12Z)
• Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond [209.35045331678043]
  We introduce a "levels x laws" taxonomy organized along two axes.<n>The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence.<n>We synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2026-04-24T17:48:47Z)
• DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA [72.9197085473598]
  DIAL is a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck.<n>A VLM-based System-2 performs latent world modeling by latent visual foresight within the VLM's native feature space.<n>A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions.
  arXiv  Detail & Related papers  (2026-03-31T15:02:27Z)
• AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation [12.592383721590402]
  We introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy)<n>Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments.<n>AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-23T17:12:25Z)
• DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI [84.9072161615971]
  We present DM0, an Embodied-Native Vision-Language-Action framework designed for Physical AI.<n>Our methodology follows a comprehensive three-stage pipeline: Pretraining, Mid-Training, and Post-Training.<n>Experiments on the RoboChallenge benchmark demonstrate that DM0 achieves state-of-the-art performance in both Specialist and Generalist settings.
  arXiv  Detail & Related papers  (2026-02-16T17:59:16Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Bridge Thinking and Acting: Unleashing Physical Potential of VLM with Generalizable Action Expert [60.88976842557026]
  Vision-Language Models (VLM) have demonstrated impressive planning and reasoning capabilities.<n>Recent dual-system approaches attempt to decouple "thinking" from "acting"<n>We introduce a framework centered around a generalizable action expert.
  arXiv  Detail & Related papers  (2025-10-04T18:33:27Z)
• Hierarchical Language Models for Semantic Navigation and Manipulation in an Aerial-Ground Robotic System [8.88014241557266]
  Heterogeneous multirobot systems show great potential in complex tasks requiring coordinated hybrid cooperation.<n>Existing methods that rely on static or task-specific models often lack generalizability across diverse tasks and dynamic environments.<n>We propose a hierarchical multimodal framework that integrates a prompted large language model (LLM) with a fine-tuned vision-language model (VLM)
  arXiv  Detail & Related papers  (2025-06-05T13:27:41Z)
• Human-Object Interaction via Automatically Designed VLM-Guided Motion Policy [30.43930233035367]
  We introduce the first unified physics-based HO framework that leverages Vision-Language Models (VLMs)<n>We introduce VLM-Guided Relative Movement Dynamics (RMD), a fine-temporal bipartite motion representation that automatically constructs goal states and reward functions for reinforcement learning.<n>To support our methodology, we present Interplay, a novel dataset with thousands of long-horizon static and dynamic interaction plans.
  arXiv  Detail & Related papers  (2025-03-24T05:18:04Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Model-Based Reinforcement Learning with Multi-Task Offline Pretraining [59.82457030180094]
  We present a model-based RL method that learns to transfer potentially useful dynamics and action demonstrations from offline data to a novel task. The main idea is to use the world models not only as simulators for behavior learning but also as tools to measure the task relevance. We demonstrate the advantages of our approach compared with the state-of-the-art methods in Meta-World and DeepMind Control Suite.
  arXiv  Detail & Related papers  (2023-06-06T02:24:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.