Fugu-MT: arxivの論文翻訳

S-Agent 2.0: Spatial Reasoning with a General Coding Agent

Authors Rui Gong, Ruoyu Wang, Yalun Dai, Hao Li, Sidi Yang, Qihao Zhao, Hongyu Li, Zeyang Guan, Fangzhou Hong, Zhaoxi Chen, Kim-Hui Yap, Ziwei Liu
Affiliations The University of Hong Kong / Nanyang Technological University / Ropedia
Categories Method / Coding Agent / Spatial reasoning with coding capabilities, Evaluation / Benchmarking / Evaluation on spatial reasoning benchmarks, Application / Spatial Reasoning / Spatial reasoning agent with external tools
License CC BY 4.0

Abstract Overview

S-Agent 2.0 is a spatial reasoning system built by equipping an off-the-shelf general coding agent with a command-line perception toolkit. The paper argues that prior agentic vision-language systems often obtain correct geometric evidence but fail to convert it into correct conclusions because acquisition, memory, and interpretation are not tightly linked. To address this, the toolkit is designed to close the evidence loop by storing observations in a persistent scene memory, allowing both tool-derived and model-derived instances to share the same geometric representation, and returning measurements with plain-language spatial interpretations. The approach preserves the flexibility of general coding agents while grounding reasoning in tool-generated geometry, and it is evaluated on multi-view and video spatial reasoning benchmarks.

Novelty

The distinctive contribution is the reframing of spatial reasoning as an evidence-loop problem and implementing it through a stock general coding agent rather than a specialized spatial harness. Methodologically, the novelty lies in the interface design combining persistent scene memory, two-way model-and-tool perception write access, and answer-ready interpreted readouts in spatial language.

Results

On MMSI-Bench and ReVSI, the toolkit improves the same general coding agent without tools by +13.9 and +20.5 points with Qwen3.8-27B, and by +11.2 and +9.6 points with Gemini-3.8-Flash. With tools, S-Agent 2.0 achieves 66.5 on MMSI-Bench and 82.3 on ReVSI with Qwen3.8-27B, outperforming SpatialClaw on the same backbone, and reaches 74.1 and 90.5 respectively with Gemini-3.8-Flash. Ablations demonstrate that removing the model-perception interface, scene memory, or interpreted readouts degrades performance, with the largest drop resulting from removing the model-perception interface.

Key Points

  1. S-Agent 2.0 equips a stock general coding agent with a command-line perception toolkit rather than relying on a specialized spatial reasoning harness.
  2. The framework closes the evidence loop by integrating persistent scene memory, shared geometric grounding for tool and model perception, and plain-language interpreted readouts.
  3. The method outperforms specialized baselines such as SpatialClaw across multi-view and video spatial reasoning benchmarks while transferring across coding harnesses including OpenHands, Codex, and Claude Code.

References

This page was created using generative AI such as GPT-5, Claude Opus 4, Gemini 3, Gemini 3.1 Flash Image, and their higher-end successor versions. No guarantee can be made regarding its contents.