OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation

• URL: http://arxiv.org/abs/2605.25829v1
• Date: Mon, 25 May 2026 13:28:33 GMT
• Title: OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation
• Abstract summary: We propose a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ trajectory prediction.<n>OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor.<n>OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization.
• Score: 41.754765514902736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent vision-language-action (VLA) models and world action models (WAMs) advance robotic manipulation by enriching intermediate representations with auxiliary spatial features or future visual-state prediction. However, these representations largely remain within the observation space and do not share the rigid-body geometry of the action space, forcing the action decoder to implicitly recover this geometry. We propose OASIS, a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ end-effector trajectory prediction. OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor that produces a camera-frame end-effector trajectory. Conditioned on the predictor's pose-supervised hidden states, the action decoder generates action chunks consistent with rigid-body motion. Across simulation and real-world experiments, OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization. Our project page is available at https://npuhandsome.github.io/OASIS_web.

Related papers

• GeomVLA: Unifying Scene, Motion, and Action in 3D [19.277341680583834]
  We present GeomVLA, a model that unifies perception, latent scene motion prediction, and action generation within a shared robot-centric 3D coordinate frame.<n>Our approach lifts pretrained VLM features into spatially grounded 3D scene tokens using depth and camera calibration.<n>We introduce a 3D Scene Trajectory Denoiser, a task-conditioned module that learns a latent representation of how scene points are expected to move in 3D.
  arXiv  Detail & Related papers  (2026-09-12T08:51:52Z)
• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN [53.419929910272735]
  We present WNM-3D, a generative World Navigation Model with 3D scene conditioning for continuous VLN.<n>To consolidate past observations into persistent scene context, a frozen feed-forward geometry encoder extracts geometry-aware representations from the monocular egocentric RGB history.<n>Through block-causal attention, this prefix conditions every future video-action block, providing a shared geometric context for both future-view and action generation.
  arXiv  Detail & Related papers  (2026-08-07T14:29:00Z)
• $μ_0$: A Scalable 3D Interaction-Trace World Model [54.38434730157622]
  We present a scalable world model based on 3D traces.<n>Our TraceExtract system automatically extracts 3D supervision.<n>These results establish 3D traces as a scalable and transferable representation for cross-embodiment manipulation.
  arXiv  Detail & Related papers  (2026-06-11T17:59:56Z)
• GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation [55.730453997808006]
  textbfGaussianDream is a feed-forward 3D Gaussian world-model plug-in that turns robot trajectories into structured spatialtemporal supervision.<n>Experiments on LIBERO, RoboCasa Human-50, and real-robot tasks demonstrate strong and highly competitive performance.
  arXiv  Detail & Related papers  (2026-05-20T05:51:30Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation [30.24537271257149]
  STARRY is a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation.<n>On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings.<n>These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation.
  arXiv  Detail & Related papers  (2026-04-29T16:13:39Z)
• Universal Pose Pretraining for Generalizable Vision-Language-Action Policies [83.39008378156647]
  Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency.<n>We propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors.<n>Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment.
  arXiv  Detail & Related papers  (2026-02-23T11:00:08Z)
• FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation [23.19464039872024]
  FlowHOI is a flow-matching framework that generates temporally coherent HOI sequences.<n>We show that FlowHOI achieves the highest action recognition accuracy and a 1.7$times$ higher physics simulation success rate.
  arXiv  Detail & Related papers  (2026-02-13T20:46:08Z)
• GeoPredict: Leveraging Predictive Kinematics and 3D Gaussian Geometry for Precise VLA Manipulation [26.632472450402947]
  Vision-Language-Action (VLA) models achieve strong generalization in robotic manipulation but remain largely reactive and 2D-centric.<n>We propose GeoPredict, a geometry-aware VLA framework that augments a continuous-action policy with predictive kinematic and geometric priors.<n>Experiments on RoboCasa Human-50, LIBERO, and real-world manipulation tasks show that GeoPredict consistently outperforms strong VLA baselines.
  arXiv  Detail & Related papers  (2025-12-18T17:51:42Z)
• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [57.942404069484134]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.<n>Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.<n>We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.