Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics

• URL: http://arxiv.org/abs/2604.21017v2
• Date: Wed, 29 Apr 2026 01:10:46 GMT
• Title: Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics
• Abstract summary: We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date.<n> GR00T-H is the first open foundation vision-language-action model for medical robotics.<n>We train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation.
• Score: 131.25998619318702
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 49 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond.

Related papers

• SurgVIL: Scaling Surgical Robot Imitation Learning with Open-source Surgical Videos [3.1495623798952823]
  SurgVIL is a framework for scaling surgical robot imitation learning using open-source surgical videos.<n>We evaluate SurgVIL on two da Vinci robot tasks: needle pick-up and cholecystectomy cutting.
  arXiv  Detail & Related papers  (2026-08-17T03:33:03Z)
• MiDAS: A Multimodal Data Acquisition System and Dataset for Robot-Assisted Minimally Invasive Surgery [4.287077788799387]
  MiDAS is a platform-agnostic system for time-synchronized, non-invasive multimodal data acquisition across surgical robotic platforms.<n>We validated MiDAS on the open-source Raven-II and the clinical da Vinci Xi by collecting multimodal datasets of peg transfer and hernia repair suturing tasks.
  arXiv  Detail & Related papers  (2026-02-12T20:56:15Z)
• SurgWorld: Learning Surgical Robot Policies from Videos via World Modeling [19.99022199561975]
  SurgWorld is a world model designed for surgical physical AI.<n>SurgeWorld generates diverse, generalizable and realistic surgery videos.<n>We are the first to use an inverse dynamics model to infer pseudokinematics from synthetic surgical videos.
  arXiv  Detail & Related papers  (2025-12-29T03:03:00Z)
• A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI [70.06771291117965]
  We introduce Biomedica, an open-source dataset derived from the PubMed Central Open Access subset.<n>Biomedica contains over 6 million scientific articles and 24 million image-text pairs.<n>We provide scalable streaming and search APIs through a web server, facilitating seamless integration with AI systems.
  arXiv  Detail & Related papers  (2025-03-26T05:56:46Z)
• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots [133.23509142762356]
  General-purpose robots need a versatile body and an intelligent mind.<n>Recent advancements in humanoid robots have shown great promise as a hardware platform for building generalist autonomy.<n>We introduce GR00T N1, an open foundation model for humanoid robots.
  arXiv  Detail & Related papers  (2025-03-18T21:06:21Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• General-purpose foundation models for increased autonomy in robot-assisted surgery [4.155479231940454]
  This perspective article aims to provide a path toward increasing robot autonomy in robot-assisted surgery. We argue that surgical robots are uniquely positioned to benefit from general-purpose models and provide three guiding actions toward increased autonomy in robot-assisted surgery.
  arXiv  Detail & Related papers  (2024-01-01T06:15:16Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• RT-1: Robotics Transformer for Real-World Control at Scale [98.09428483862165]
  We present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.
  arXiv  Detail & Related papers  (2022-12-13T18:55:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.