GeomVLA: Unifying Scene, Motion, and Action in 3D

• URL: http://arxiv.org/abs/2609.13812v1
• Date: Sat, 12 Sep 2026 08:51:52 GMT
• Title: GeomVLA: Unifying Scene, Motion, and Action in 3D
• Abstract summary: We present GeomVLA, a model that unifies perception, latent scene motion prediction, and action generation within a shared robot-centric 3D coordinate frame.<n>Our approach lifts pretrained VLM features into spatially grounded 3D scene tokens using depth and camera calibration.<n>We introduce a 3D Scene Trajectory Denoiser, a task-conditioned module that learns a latent representation of how scene points are expected to move in 3D.
• Score: 19.277341680583834
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GeomVLA, a Vision-Language-Action (VLA) model that unifies perception, latent scene motion prediction, and action generation within a shared robot-centric 3D coordinate frame. Our approach lifts pretrained VLM features into spatially grounded 3D scene tokens using depth and camera calibration, while retaining the semantic representations learned during VLM pretraining. We further introduce a 3D Scene Trajectory Denoiser, a task-conditioned module that learns a latent representation of how scene points are expected to move in 3D. Rather than executing the predicted trajectory as an open-loop plan, GeomVLA extracts intermediate motion tokens from the trajectory denoiser and uses them to condition a 3D flow-based action denoiser through geometry-aware attention. GeomVLA achieves state-of-the-art performance on CALVIN, competitive performance on LIBERO and RoboTwin2.0, and outperforms strong baselines in real-world manipulation settings without robot-action pretraining. Extensive ablations show that future-motion reasoning alone is insufficient: the primary gains are associated with maintaining geometric consistency among scene representation, motion prediction, and robot actions throughout the perception-to-action pipeline.

Related papers

• WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN [53.419929910272735]
  We present WNM-3D, a generative World Navigation Model with 3D scene conditioning for continuous VLN.<n>To consolidate past observations into persistent scene context, a frozen feed-forward geometry encoder extracts geometry-aware representations from the monocular egocentric RGB history.<n>Through block-causal attention, this prefix conditions every future video-action block, providing a shared geometric context for both future-view and action generation.
  arXiv  Detail & Related papers  (2026-08-07T14:29:00Z)
• Ego-Human Motion Prediction with 3D-Aware LLM [51.5064259568401]
  Ego3DLM is a tool for predicting human motion from an egocentric perspective.<n>It decodes past pose, future pose, past narration, and future narration in a single autoregressive pass.<n>Ego3DLM achieves state-of-the-art performance across future motion prediction, past motion tracking, and motion description.
  arXiv  Detail & Related papers  (2026-07-08T04:51:24Z)
• Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation [76.16065370700488]
  We introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud reasoning and enables temporally coherent action generation.<n>We show that Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods.
  arXiv  Detail & Related papers  (2026-07-07T17:59:47Z)
• WAM4D: Fast 4D World Action Model via Spatial Register Tokens [52.16661403052964]
  We present WAM4D, a fast 4D world action model that uses lightweight spatial register tokens as training-time future-depth readouts.<n>We show that WAM4D improves spatial consistency and achieves competitive action prediction while maintaining efficient inference.
  arXiv  Detail & Related papers  (2026-06-12T02:49:34Z)
• 3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training [19.5808550016589]
  3D geometry perception and 3D spatial reasoning are distinct capabilities that can be disentangled and injected at different feature hierarchies.<n>We propose a 3D-thinking-guided co-training framework that enables vision-language-action (VLA) models to perform 3D spatial reasoning implicitly during action prediction.
  arXiv  Detail & Related papers  (2026-06-03T04:34:07Z)
• OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation [41.754765514902736]
  We propose a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ trajectory prediction.<n>OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor.<n>OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-05-25T13:28:33Z)
• StemVLA:An Open-Source Vision-Language-Action Model with Future 3D Spatial Geometry Knowledge and 4D Historical Representation [6.0744834626758495]
  StemVLA is a novel framework that explicitly incorporates both future-oriented 3D spatial knowledge and historical 4D representations into action prediction.<n>We show that StemVLA significantly improves long-horizon task success and state-of-the-art performance on the CALVIN ABC-D benchmark [46], achieving an average sequence length of XXX.
  arXiv  Detail & Related papers  (2026-02-27T06:43:37Z)
• Universal Pose Pretraining for Generalizable Vision-Language-Action Policies [83.39008378156647]
  Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency.<n>We propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors.<n>Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment.
  arXiv  Detail & Related papers  (2026-02-23T11:00:08Z)
• CLAMP: Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining [4.039082584778385]
  We introduce Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining (CLAMP)<n>From the merged point cloud computed from RGB-D images and camera extrinsics, we re-render multi-view four-channel image observations with depth and 3D coordinates.<n>The pre-trained encoders learn to associate the 3D geometric and positional information of objects with robot action patterns.
  arXiv  Detail & Related papers  (2026-01-31T23:32:54Z)
• REACT3D: Recovering Articulations for Interactive Physical 3D Scenes [96.27769519526426]
  REACT3D is a framework that converts static 3D scenes into simulation-ready interactive replicas with consistent geometry.<n>We achieve state-of-the-art performance on detection/segmentation and articulation metrics across diverse indoor scenes.
  arXiv  Detail & Related papers  (2025-10-13T12:37:59Z)
• GAF: Gaussian Action Field as a 4D Representation for Dynamic World Modeling in Robotic Manipulation [47.471097712217386]
  Existing approaches typically follow either a Vision-to-Action (V-A) paradigm, predicting actions directly from visual inputs, or a Vision-to-3D-to-Action (V-3D-A) paradigm, leveraging intermediate 3D representations.<n>In this paper, we adopt a V-4D-A framework that enables direct action reasoning from motion-aware 4D representations via a Gaussian Action Field (GAF)<n>Experiments demonstrate significant improvements, with achieving +11.5385 dB PSNR, +0.3864 SSIM and -0.5574 LPIPS improvements in reconstruction quality, while boosting the average +7.3% success
  arXiv  Detail & Related papers  (2025-06-17T02:55:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.