The Robot Data Factory

• URL: http://arxiv.org/abs/2609.16705v1
• Date: Tue, 15 Sep 2026 06:30:11 GMT
• Title: The Robot Data Factory
• Abstract summary: We introduce the Robot Data Factory (RDF), a mission-driven infrastructure for continuously generating, validating, benchmarking, and reusing experience.<n>RDF organizes heterogeneous robots and environment-specific training grounds through reproducible missions, skill curricula, synchronized multimodal sensing, external ground truth, an agentic robot network, data pipelines, and living benchmarks.<n>We further formalize robot experience and its quality, introduce a mission-task-skill-episode-dataset-benchmark-capability hierarchy, and derive quantitative scaling laws and an algorithmic procedure connecting robot fleet size, sensor rates, storage, learning representations, tokenization, training compute
• Score: 35.46679263015648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical AI requires more than increasingly large robot datasets: intelligent robots acquire knowledge through continuous interaction with the physical world. We argue that the defining scientific resource of Physical AI is therefore not raw robot data alone, but robot experience - physically grounded interaction whose observations, actions, embodiment, context, and outcomes preserve the perception-action-consequence loop. We introduce the Robot Data Factory (RDF), a mission-driven infrastructure and methodology for continuously generating, validating, benchmarking, and reusing such experience. RDF organizes heterogeneous robots and environment-specific training grounds through reproducible missions, skill curricula, synchronized multimodal sensing, external ground truth, an agentic robot network, data pipelines, and living benchmarks. Rather than treating datasets as static end products, RDF implements a closed Deploy-Measure-Learn-Repeat cycle in which validated physical experience supports world models, vision-language-action models, embodied policies, digital twins, and subsequent robot deployment. We further formalize robot experience and its quality, introduce a mission-task-skill-episode-dataset-benchmark-capability hierarchy, and derive quantitative scaling laws and an algorithmic synthesis procedure connecting robot fleet size, sensor rates, storage, learning representations, tokenization, training compute, inference, and latency to Embodied-AI cluster requirements. The framework is instantiated in three complementary physical training grounds for domestic, environmental, and energy applications. RDF thus reframes robot data generation as a continuous scientific production process and provides a pathway toward reproducible, scalable, and eventually federated infrastructure for Physical AI.

Related papers

• RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills [90.48218237655708]
  RoboSynChallenge aims to foster the development of broadly capable, data-efficient, and adaptable manipulation systems.<n>Challenge integrates large-scale synthetic data generation with standardized real-world robotic evaluation.
  arXiv  Detail & Related papers  (2026-08-12T04:29:00Z)
• Data Standards for Humanoid Robotics: The Missing Infrastructure for Physical AI [24.21963728682223]
  We argue that data standards are becoming foundational infrastructure for Physical AI.<n>As AI moves from screens into bodies, data standards must evolve from organizing digital information to structuring physical interaction.
  arXiv  Detail & Related papers  (2026-06-18T04:10:16Z)
• DataLadder: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid [63.45560198934907]
  Generalist robot policies require trustworthy evaluation and robot-usable training data.<n>We present DataLadder, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation.<n>We package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud.
  arXiv  Detail & Related papers  (2026-06-15T14:21:35Z)
• Generative Simulation for Policy Learning in Physical Human-Robot Interaction [11.978357519356537]
  We introduce a zero-shot "text2sim2real" generative simulation framework that automatically synthesizes diverse pHRI scenarios.<n>We utilize this framework to autonomously collect large-scale synthetic demonstration datasets.<n>Our learned policies successfully achieve zero-shot sim-to-real transfer, attaining success rates exceeding 80%.
  arXiv  Detail & Related papers  (2026-04-09T18:00:05Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• Curate, Connect, Inquire: A System for Findable Accessible Interoperable and Reusable (FAIR) Human-Robot Centered Datasets [2.9620440248101967]
  The rapid growth of AI in robotics has amplified the need for high-quality, reusable datasets.<n>The landscape of open data in the field is uneven due to a lack of curation standards and consistent publication practices.<n>This paper presents a curation and access system with two main contributions.
  arXiv  Detail & Related papers  (2025-05-30T20:48:32Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents [109.3804962220498]
  AutoRT is a system to scale up the deployment of operational robots in completely unseen scenarios with minimal human supervision. We demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both teleoperation and autonomous robot policies. We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
  arXiv  Detail & Related papers  (2024-01-23T18:45:54Z)
• RT-1: Robotics Transformer for Real-World Control at Scale [98.09428483862165]
  We present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.
  arXiv  Detail & Related papers  (2022-12-13T18:55:15Z)
• SAGCI-System: Towards Sample-Efficient, Generalizable, Compositional, and Incremental Robot Learning [41.19148076789516]
  We introduce a systematic learning framework called SAGCI-system towards achieving the above four requirements. Our system first takes the raw point clouds gathered by the camera mounted on the robot's wrist as the inputs and produces initial modeling of the surrounding environment represented as a URDF. The robot then utilizes the interactive perception to interact with the environments to online verify and modify the URDF.
  arXiv  Detail & Related papers  (2021-11-29T16:53:49Z)
• HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing [1.3678064890824186]
  The Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates.
  arXiv  Detail & Related papers  (2021-10-20T00:41:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.