ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies

• URL: http://arxiv.org/abs/2606.16826v1
• Date: Mon, 15 Jun 2026 15:08:42 GMT
• Title: ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies
• Abstract summary: Generalist manipulation policies are increasingly presented as foundation models for robotic control.<n>ATOM-Bench is a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies.
• Score: 18.55273532676497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalist manipulation policies are increasingly presented as foundation models for robotic control, but their real-world generalization remains difficult to diagnose. A policy may succeed on demonstrated tasks while still failing to execute fine-grained atomic skills or recombine learned skills in new task structures. We introduce \textbf{ATOM-Bench}, a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies. ATOM-Bench factorizes tabletop manipulation into motor atoms and instruction atoms, and contains 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. We collect 3,000 human demonstrations for atomic fine-tuning and release both the demonstration data and evaluation rollout data to support reproducible real-world evaluation. Policies are fine-tuned on atomic tasks and evaluated on both atomic skill acquisition and held-out compositional tasks. We further introduce Atomic Score (AS) and Compositional Failure Share (CFS) to distinguish failures caused by weak atomic skills from failures caused by limited compositional reuse. Through 2,700 physical rollouts on five representative manipulation policies, we find that current policies can acquire simple instruction-grounding skills, but still struggle with fine-grained motor atoms, counting, and logical filtering. More importantly, strong atomic performance does not reliably transfer to held-out compositional tasks. ATOM-Bench provides a diagnostic testbed for studying whether failures arise from weak motor execution, poor instruction grounding, or limited compositional reuse.

Related papers

• RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies [121.35821705538768]
  Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities.<n>We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies.<n>RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities.
  arXiv  Detail & Related papers  (2026-07-05T17:58:02Z)
• EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents [75.01735520608075]
  Existing benchmarks mostly test literal Theory of Mind (ToM) by asking direct belief questions.<n>We introduce EnactToM, an evolving benchmark of 300 embodied multi-agent tasks set in a 3D household.<n>On the hard split, all seven evaluated frontier models score 0.0% Pass3 on functional task completion, while averaging 45.0% on literal belief probes.
  arXiv  Detail & Related papers  (2026-05-11T00:04:19Z)
• Scaling Coding Agents via Atomic Skills [23.614587850545988]
  We first formalize five fundamental atomic skills, code localization, code editing, unit-test generation, issue reproduction, and code review.<n>Compared with composite coding tasks, these atomic skills are more general and composable.<n>In this manner, atomic skills are consistently improved without negative interference or trade-offs between them.
  arXiv  Detail & Related papers  (2026-04-06T16:36:21Z)
• AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robots [92.18094199070693]
  Real-world robotic tasks often involve long-horizon, multi-step problem-solving and require generalization for continual skill acquisition.<n>We propose AtomicVLA, a unified planning-and-execution framework that jointly generates task-level plans, atomic skill abstractions, and fine-grained actions.<n>In simulation, AtomicVLA outperforms $_0$ by 2.4% on LIBERO, 10% on LIBERO-LONG, and outperforms $_0$ and $_0.5$ by 0.22 and 0.25 in average task length on CALVIN
  arXiv  Detail & Related papers  (2026-03-08T14:18:56Z)
• Learning Semantic Atomic Skills for Multi-Task Robotic Manipulation [34.73582639920571]
  AtomSkill is a novel multi-task imitation learning framework.<n>It learns and leverages a structured Atomic Skill Space for composable robot manipulation.<n>It consistently outperforms state-of-the-art methods across diverse manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-20T13:46:08Z)
• From Atomic to Composite: Reinforcement Learning Enables Generalization in Complementary Reasoning [83.94543243783285]
  We study Complementary Reasoning, a complex task that requires integrating internal parametric knowledge with external contextual information.<n>We find that RL acts as a reasoning synthesizer rather than a probability amplifier.
  arXiv  Detail & Related papers  (2025-12-01T18:27:25Z)
• RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation [32.080769025457926]
  We present RoboEval, a simulation benchmark and structured evaluation framework designed to reveal the limitations of current bimanual manipulation policies.<n>RoboEval introduces a suite of tiered, semantically grounded tasks that systematically challenge spatial, physical, and coordination capabilities.<n>We find that behavioral metrics correlate with success in over half of task-metric pairs, and remain informative even when binary success saturates.
  arXiv  Detail & Related papers  (2025-07-01T05:33:16Z)
• Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models? [68.72260770171212]
  We propose a paradigm of Self-structured Chain of Thought (SCoT), which is composed of minimal semantic atomic steps.<n>Our method can not only generate cognitive CoT structures for various complex tasks but also mitigates the phenomenon of overthinking.<n>We conduct extensive experiments to show that the proposed AtomThink significantly improves the performance of baseline MLLMs.
  arXiv  Detail & Related papers  (2025-03-08T15:23:47Z)
• AtomR: Atomic Operator-Empowered Large Language Models for Heterogeneous Knowledge Reasoning [49.24235059299745]
  We introduce AtomR, a framework for large language models to conduct accurate heterogeneous knowledge reasoning at the atomic level.<n>AtomR decomposes a complex question into a reasoning tree where each leaf node corresponds to an atomic knowledge operator.<n>In the reasoning execution stage, AtomR executes each atomic knowledge operator, which flexibly selects, retrieves, and operates atomic level knowledge from heterogeneous sources.
  arXiv  Detail & Related papers  (2024-11-25T15:35:51Z)
• Laying the Foundation First? Investigating the Generalization from Atomic Skills to Complex Reasoning Tasks [40.7766635942194]
  We propose a probing framework to investigate whether the atomic skill can spontaneously generalize to complex reasoning tasks. We then introduce a hierarchical curriculum learning training strategy to achieve better skill generalization. By leveraging hierarchical curriculum learning, we successfully induce generalization, significantly improve the performance of open-source LMs on complex reasoning tasks.
  arXiv  Detail & Related papers  (2024-03-14T15:20:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.