DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation

• URL: http://arxiv.org/abs/2608.01381v1
• Date: Sun, 02 Aug 2026 17:02:48 GMT
• Title: DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation
• Abstract summary: We present DreamTrajectory, a trajectory-guided framework for language-conditioned mobile manipulation.<n>On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%.<n>On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.
• Score: 33.046851310859495
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mobile manipulation requires a robot to coordinate base and arm motion under continuously changing viewpoints and contact conditions, within an action space far larger than that of fixed-base manipulation. Existing Vision-Language-Action (VLA) policies are limited in two respects. (i)They map observations directly to whole-body action chunks, searching this large action space without an explicit task-space motion plan, which makes coordinated base--arm prediction imprecise. (ii)They execute the predicted chunk open-loop, without checking whether the actions can realize the motion the policy intended, so control errors and unmodeled contacts accumulate into a gap between planned and realized motion. We present DreamTrajectory, a trajectory-guided framework for language-conditioned mobile manipulation that introduces one component for each limitation. Addressing(i), DreamTrajectory jointly predicts an intention-level end-effector trajectory and a whole-body action chunk in a single action expert, so that the trajectory explicitly guides base--arm action generation instead of remaining implicit. Addressing(ii), a lightweight trajectory world model predicts the trajectory that a candidate action chunk would induce, and a test-time search--predict--score procedure selects the candidate best aligned with the planned trajectory. On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%, with the largest gains on contact-rich articulated-object tasks. On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.

Related papers

• Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model [7.384043308834925]
  Visual goal and dynamics prediction can provide language-conditioned robot policies with both a target outcome and a representation of action-dependent scene changes.<n>Dynin-Robotics implements this formulation on Dynin- Omni, representing language, visual observations, goals, and actions as discrete tokens.<n>We continually pretrain the model on approximately 1.33 million trajectories from 48 Open X-Embodiment datasets and adapt it separately to downstream domains.
  arXiv  Detail & Related papers  (2026-09-11T16:49:26Z)
• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting [75.76075128622837]
  We propose a two-stage framework that introduces Explicit Manipulation Planning as an Intermediate Representation for Egocentric hand-motion forecasting.<n>A motion generator synthesizes future bimanual hand motions conditioned on frozen planner representations.<n>Our method achieves state-of-the-art forecasting accuracy, with an MPJPE of 84.53 mm and a finger-relative error of 38.97mm.
  arXiv  Detail & Related papers  (2026-08-23T14:58:31Z)
• ABot-M0.5: Unified Mobility-and-Manipulation World Action Model [27.262271208923995]
  ABot-M0.5 is built on the insight that mobile manipulation requires alignment at three levels: temporal granularity, action space, and train-test consistency.<n> Experiments on challenging mobile and fine-grained manipulation benchmarks demonstrate that ABot-M0.5 achieves state-of-the-art performance in both long-horizon task success and finegrained control accuracy.
  arXiv  Detail & Related papers  (2026-07-01T09:21:20Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models [19.75611749501909]
  Vision-Ground-action (VLA) models have shown strong promise for robotic manipulation.<n>But their reliability at test time remains limited by one-shot action prediction.<n>We present VeriSpace, a 3D-aware verifier for test-time action selection in VLA systems.
  arXiv  Detail & Related papers  (2026-06-09T08:31:59Z)
• STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation [30.24537271257149]
  STARRY is a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation.<n>On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings.<n>These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation.
  arXiv  Detail & Related papers  (2026-04-29T16:13:39Z)
• Lifting Embodied World Models for Planning and Control [59.09016913513998]
  We train a lightweight policy that maps high-level actions to sequences of low-level joint actions.<n>We instantiate this framework for a human-like embodiment, defining the high-level action space as a small set of 2D waypoints.<n>We show that the lifted world model substantially outperforms searching directly in low-level joint space.
  arXiv  Detail & Related papers  (2026-04-28T23:59:19Z)
• ActionSink: Toward Precise Robot Manipulation with Dynamic Integration of Action Flow [93.00917887667234]
  This paper introduces a novel robot manipulation framework, i.e., ActionSink, to pave the way toward precise action estimations.<n>As the name suggests, ActionSink reformulates the actions of robots as action-caused optical flows from videos, called "action flow"<n>Our framework outperformed prior SOTA on the LIBERO benchmark by a 7.9% success rate, and obtained nearly an 8% accuracy gain on the challenging long-horizon visual task LIBERO-Long.
  arXiv  Detail & Related papers  (2025-08-05T08:46:17Z)
• Motion Transformer with Global Intention Localization and Local Movement Refinement [103.75625476231401]
  Motion TRansformer (MTR) models motion prediction as the joint optimization of global intention localization and local movement refinement. MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges.
  arXiv  Detail & Related papers  (2022-09-27T16:23:14Z)
• Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction [81.94175022575966]
  We introduce the task of action-driven human motion prediction. It aims to predict multiple plausible future motions given a sequence of action labels and a short motion history.
  arXiv  Detail & Related papers  (2022-05-31T08:38:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.