Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation

• URL: http://arxiv.org/abs/2608.23138v1
• Date: Mon, 24 Aug 2026 11:43:49 GMT
• Title: Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation
• Abstract summary: We present Pointing-VLA, a typed hidden-state spatial readout built on Embodied-R1.<n>Pointing-VLA achieves SOTA performance on Bridge/WidowX, averaging 72.9% across the evaluated four-task set.
• Score: 5.156481604986546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models often expose spatial grounding through autoregressive text coordinates or opaque action tokens, creating brittle interfaces between multimodal reasoning and robot execution. We present Pointing-VLA, a typed hidden-state spatial readout built on Embodied-R1. Geometry-specific heads predict normalized points, object-functional grounding (OFG) heatmaps, and visual trajectories without serializing geometry as text. For the evaluated Bridge/WidowX and physical pick-place deployments, an explicit execution contract assigns PICK to source-conditioned OFG and PLACE to Pointing, providing direct stage-aligned spatial targets. Pointing-VLA achieves SOTA performance on Bridge/WidowX, averaging 72.9\% across the evaluated four-task set without Bridge-specific finetuning under collision-enabled CuRobo execution. Pointing and OFG show complementary strengths across native and cross-dataset evaluations. The OFG/contact readout transfers to NORA-1.5, preserving or improving success while reducing recorded controller time by more than 20$\times$; typed heads are also 6.68--6.90$\times$ faster than Embodied-R1 text decoding on a shared external suite. When integrated as spatial guidance for a $π_{0.5}$ action policy, Pointing-VLA raises autonomous real-robot success from 52.7\% to 80.7\% across three visual contexts. These results establish typed spatial readouts as an efficient, inspectable interface between embodied reasoning and robot execution.

Related papers

• Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision [26.793499191581315]
  Cross-embodiment transfer in vision-language-action (VLA) models remains challenging.<n>We present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Reason (ECoT) supervision to align cross-embodiment representations.
  arXiv  Detail & Related papers  (2026-06-29T16:48:48Z)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning [99.49739831030388]
  We study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.<n>We propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface.
  arXiv  Detail & Related papers  (2026-06-11T17:59:36Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• EmbodimentSemantic: A Spatial Scene-Graph Dataset and Benchmark for Vision-Language Models on Embodied Manipulation Trajectories [16.996900088118412]
  We introduce EmbodimentSemantic, a spatial scene-graph dataset and benchmark for evaluating relational grounding in embodied manipulation.<n>The dataset includes real-world manipulation observations collected with the low-cost SO101 robot arm.<n>To provide controlled validation, we also introduce a simulator-grounded LIBERO benchmark with over 60K manipulation frames and more than 120K camera-specific scene graphs across paired third-person and wrist views.
  arXiv  Detail & Related papers  (2026-06-06T18:58:54Z)
• OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation [41.754765514902736]
  We propose a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ trajectory prediction.<n>OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor.<n>OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-05-25T13:28:33Z)
• VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models [80.1055544841585]
  Vision-Language-Action models typically map visual observations and linguistic instructions directly to robotic control signals.<n>We propose VP-VLA, a dual-system framework that decouples high-level reasoning and low-level execution via a structured visual prompting interface.<n> Experiments on the Robocasa-GR1-Tabletop benchmark and SimplerEnv simulation demonstrate that VP-VLA improves success rates by 5% and 8.3%.
  arXiv  Detail & Related papers  (2026-03-23T14:08:58Z)
• Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models [2.5338045371474816]
  Vision-Language-Action (VLA) models combine perception, language, and motor control in a single architecture.<n>We apply activation injection, sparse autoencoders, and linear probes to six models spanning 80M--7B parameters across 394,000+ rollout episodes on four benchmarks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:55Z)
• SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulation [65.6201974979119]
  We propose SemanticVLA, a novel VLA framework that performs Semantic-Hierarchical Sparsification and Enhancement for Efficient Robotic Manipulation.<n>SemanticVLA surpasses OpenVLA on LIBERO benchmark by 21.1% in success rate, while reducing training cost and inference latency by 3.0-fold and 2.7-fold.
  arXiv  Detail & Related papers  (2025-11-13T17:24:37Z)
• Detect Anything via Next Point Prediction [51.55967987350882]
  Rex- Omni is a 3B-scale MLLM that achieves state-of-the-art object perception performance.<n>On benchmarks like COCO and LVIS, Rex- Omni attains performance comparable to or exceeding regression-based models.
  arXiv  Detail & Related papers  (2025-10-14T17:59:54Z)
• You Only Estimate Once: Unified, One-stage, Real-Time Category-level Articulated Object 6D Pose Estimation for Robotic Grasping [119.41166438439313]
  YOEO is a single-stage method that outputs instance segmentation and NPCS representations in an end-to-end manner.<n>We use a unified network to generate point-wise semantic labels and centroid offsets, allowing points from the same part instance to vote for the same centroid.<n>We also deploy our synthetically-trained model in a real-world setting, providing real-time visual feedback at 200Hz.
  arXiv  Detail & Related papers  (2025-06-06T03:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.