SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning
Abstract Overview
This paper studies how the action interface of a tool-augmented agent affects open-ended spatial reasoning. It proposes SpatialClaw, a training-free framework in which a VLM-backed agent writes and executes Python code step by step in a persistent kernel, rather than relying on one-shot code execution or fixed structured tool calls. The workspace keeps intermediate states such as masks, depth maps, reconstructions, and visualizations available across turns, allowing the agent to inspect results and revise its analysis. The framework is evaluated on 20 benchmarks covering single-image, multi-view, video, and 3D/4D spatial reasoning tasks, using six VLM backbones from the Qwen and Gemma families.
Novelty
The paper's main novelty is reframing code itself as the action interface for spatial reasoning agents, implemented as iterative cell execution in a persistent Python workspace. This distinguishes the method from prior single-pass code systems and structured tool-call agents by emphasizing stateful composition, inspection, and revision of intermediate spatial evidence.
Results
Across 20 benchmarks, SpatialClaw achieves 59.9% average accuracy and outperforms the recent spatial agent baseline by 11.2 percentage points on average. The reported gains are strongest on dynamic 4D video reasoning and multi-view tasks, and they remain consistent across six backbone models without benchmark-specific or model-specific adaptation.
Key Points
- SpatialClaw uses a persistent Python kernel so the agent can compose perception tools and scientific libraries across multiple steps while reusing intermediate variables and visual feedback.
- The method is evaluated broadly across 20 benchmarks spanning single-image, multi-view, general spatial, video spatial/4D, and general video understanding tasks.
- Ablation and interface comparisons indicate that the action-interface design itself contributes substantially to performance, with advantages over both structured tool calls and single-pass code execution.
References
- arXiv: https://arxiv.org/abs/2606.13673v1
- Fugu-MT: https://fugumt.com/fugumt/paper_check/2606.13673v1
- Hugging Face Papers: https://huggingface.co/papers/2606.13673