RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?

• URL: http://arxiv.org/abs/2609.05324v1
• Date: Fri, 04 Sep 2026 16:19:01 GMT
• Title: RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?
• Abstract summary: Vision-Language-Action (VLA) models have shown promising progress in language-conditioned robotic manipulation.<n>We introduce textbfRoboSPA (textbfRobot textbfSpatial-textbfProcedural textbfAssessment), a large-scale robotic manipulation dataset and benchmark for diagnosing embodied reasoning.
• Score: 65.16829463527377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have shown promising progress in language-conditioned robotic manipulation. However, existing datasets and benchmarks mainly evaluate task completion under predefined settings, offering limited insight into model reasoning under increasing spatial and procedural complexity. We introduce \textbf{RoboSPA} (\textbf{Robo}t \textbf{S}patial-\textbf{P}rocedural \textbf{A}ssessment), a large-scale robotic manipulation dataset and benchmark for diagnosing embodied reasoning in VLA models. \texttt{RoboSPA} focuses on two core dimensions, Fine-Grained Spatial Reasoning and Long-Horizon Procedural Planning, covering 10 task categories and 56 base tasks. Each task is instantiated across five difficulty levels, yielding 280 variants with increasing spatial ambiguity and procedural complexity. We collect 527K trajectories across multiple embodiments and diverse scenes. Beyond binary success rate, \texttt{RoboSPA} introduces diagnostic metrics for more detailed evaluation. Experiments on representative VLA models show that current systems still struggle with complex spatial relations, precise low-level execution, and memory-intensive planning. These results establish \texttt{RoboSPA} as a challenging diagnostic benchmark for developing more capable, reliable, and generalizable embodied agents. Our data and code are available at https://github.com/fanzhenxuan/RoboSPA.

Related papers

• ST-$π$: Structured SpatioTemporal VLA for Robotic Manipulation [24.44733656837927]
  We propose a structured structuredtemporal V model for robotic manipulation.<n>Our model is guided by two key designs: 1) Stemporal VLM and 2) Stemporal action expert.<n>In addition, we propose a real-world robotic dataset with annotations for fine-tuning.
  arXiv  Detail & Related papers  (2026-04-20T06:48:47Z)
• AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models [9.608633915316252]
  Vision-Language-Action (VLA) models demonstrate remarkable potential for generalizable robotic manipulation.<n>Current paradigms rely on coarse, high-level task instructions during supervised fine-tuning.<n>We propose method, the first subtask-aware VLA framework integrated with a scalable offline post-training pipeline.
  arXiv  Detail & Related papers  (2026-03-09T15:52:48Z)
• RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation [104.68774434699158]
  RoboInter Manipulation Suite is a unified resource including data, benchmarks, and models of intermediate representations for manipulation.<n>It comprises RoboInter-Tool, a lightweight GUI that enables semi-automatic annotation of diverse representations, and RoboInter-Data, a large-scale dataset containing over 230k episodes across 571 diverse scenes.<n>RoboInter-VLA offers an integrated plan-then-execute framework, supporting modular and end-to-end VLA variants.
  arXiv  Detail & Related papers  (2026-02-10T17:01:54Z)
• VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models [33.143187576673775]
  We propose a novel structured task design framework to quantify difficulty across three axes.<n>For Task Structure, VLA-Arena's 170 tasks are grouped into four dimensions: Safety, Distractor, Extrapolation, and Long Horizon.<n>We provide an end-to-end toolchain from task definition to automated evaluation and the VLA-Arena-S/M/L datasets for fine-tuning.
  arXiv  Detail & Related papers  (2025-12-27T09:40:54Z)
• Vision-Language-Action Models for Selective Robotic Disassembly: A Case Study on Critical Component Extraction from Desktops [5.567801088767209]
  High-value items like RAM modules and CPUs, as well as sensitive parts like hard disk drives, require sequential, precise, and dexterous operations.<n>Recent development of vision-language-action (VLA) models has presented an end-to-end approach for general robotic manipulation tasks.<n>In this paper, we collected a customized dataset for robotic RAM and CPU disassembly and used it to fine-tune two well-established VLA approaches.
  arXiv  Detail & Related papers  (2025-12-04T04:36:39Z)
• UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarios [63.67884284105684]
  We introduce textbfUltraHorizon, a novel benchmark that measures the foundational capabilities essential for complex real-world challenges.<n>Agents are designed in long-horizon discovery tasks where they must iteratively uncover hidden rules.<n>Our experiments reveal that LLM-agents consistently underperform in these settings, whereas human participants achieve higher scores.
  arXiv  Detail & Related papers  (2025-09-26T02:04:00Z)
• DexVLG: Dexterous Vision-Language-Grasp Model at Scale [59.5613919093295]
  There is little research on functional grasping with large models for human-like dexterous hands.<n>We introduce DexVLG, a large Vision-Language-Grasp model for Dexterous grasp pose prediction aligned with language instructions.<n>We generate a dataset of 170 million dexterous grasp poses mapped to semantic parts across 174,000 objects in simulation, paired with detailed part-level captions.
  arXiv  Detail & Related papers  (2025-07-03T16:05:25Z)
• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation [59.9896841079005]
  We introduce RoboCerebra, a benchmark for evaluating high-level reasoning in long-horizon robotic manipulation.<n>The dataset is constructed via a top-down pipeline, where GPT generates task instructions and decomposes them into subtask sequences.<n>Compared to prior benchmarks, RoboCerebra features significantly longer action sequences and denser annotations.
  arXiv  Detail & Related papers  (2025-06-07T06:15:49Z)
• RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics [67.11221574129937]
  Spatial referring is a fundamental capability of embodied robots to interact with the 3D physical world.<n>We propose RoboRefer, a 3D-aware VLM that can first achieve precise spatial understanding.<n>RoboRefer advances generalized multi-step spatial reasoning via reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2025-06-04T17:59:27Z)
• LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks [31.3295171851909]
  Real-world embodied agents face high-level goals demanding multi-step solutions.<n>Long-horizon tasks require high-level task planning and low-level motion control.<n>We introduce a new unified vision language framework for long-horizon tasks, dubbed LoHoVLA.
  arXiv  Detail & Related papers  (2025-05-31T06:01:03Z)
• HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation [54.03004125910057]
  We show that hierarchical vision-language-action models can be more effective in utilizing off-domain data than standard monolithic VLA models.<n>We show that, with the hierarchical design, the high-level VLM can transfer across significant domain gaps between the off-domain finetuning data and real-robot testing scenarios.
  arXiv  Detail & Related papers  (2025-02-08T07:50:22Z)
• LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation [38.66406497318709]
  This work focuses on the tabletop manipulation task and releases a simulation benchmark, textitLoHoRavens, which covers various long-horizon reasoning aspects spanning color, size, space, arithmetics and reference. We investigate two methods of bridging the modality gap: caption generation and learnable interface for incorporating explicit and implicit observation feedback to the LLM.
  arXiv  Detail & Related papers  (2023-10-18T14:53:14Z)
• DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment [10.322491116030825]
  DoReMi enables Detection and Recovery from Misalignments between plan and execution. Our pipeline can monitor the low-level execution and enable timely recovery if certain plan-execution misalignment occurs.
  arXiv  Detail & Related papers  (2023-07-01T12:51:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.