Curate, Connect, Inquire: A System for Findable Accessible Interoperable and Reusable (FAIR) Human-Robot Centered Datasets

• URL: http://arxiv.org/abs/2506.00220v1
• Date: Fri, 30 May 2025 20:48:32 GMT
• Title: Curate, Connect, Inquire: A System for Findable Accessible Interoperable and Reusable (FAIR) Human-Robot Centered Datasets
• Abstract summary: The rapid growth of AI in robotics has amplified the need for high-quality, reusable datasets.<n>The landscape of open data in the field is uneven due to a lack of curation standards and consistent publication practices.<n>This paper presents a curation and access system with two main contributions.
• Score: 2.9620440248101967
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid growth of AI in robotics has amplified the need for high-quality, reusable datasets, particularly in human-robot interaction (HRI) and AI-embedded robotics. While more robotics datasets are being created, the landscape of open data in the field is uneven. This is due to a lack of curation standards and consistent publication practices, which makes it difficult to discover, access, and reuse robotics data. To address these challenges, this paper presents a curation and access system with two main contributions: (1) a structured methodology to curate, publish, and integrate FAIR (Findable, Accessible, Interoperable, Reusable) human-centered robotics datasets; and (2) a ChatGPT-powered conversational interface trained with the curated datasets metadata and documentation to enable exploration, comparison robotics datasets and data retrieval using natural language. Developed based on practical experience curating datasets from robotics labs within Texas Robotics at the University of Texas at Austin, the system demonstrates the value of standardized curation and persistent publication of robotics data. The system's evaluation suggests that access and understandability of human-robotics data are significantly improved. This work directly aligns with the goals of the HCRL @ ICRA 2025 workshop and represents a step towards more human-centered access to data for embodied AI.

Related papers

• The Robot Data Factory [35.46679263015648]
  We introduce the Robot Data Factory (RDF), a mission-driven infrastructure for continuously generating, validating, benchmarking, and reusing experience.<n>RDF organizes heterogeneous robots and environment-specific training grounds through reproducible missions, skill curricula, synchronized multimodal sensing, external ground truth, an agentic robot network, data pipelines, and living benchmarks.<n>We further formalize robot experience and its quality, introduce a mission-task-skill-episode-dataset-benchmark-capability hierarchy, and derive quantitative scaling laws and an algorithmic procedure connecting robot fleet size, sensor rates, storage, learning representations, tokenization, training compute
  arXiv  Detail & Related papers  (2026-09-15T06:30:11Z)
• Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data [60.94418315274988]
  Ego2Robot supports both curated and in-the-wild videos, producing 18,561 hours of robot training data spanning 15 robot morphologies.<n>Joint pretraining on Ego2Robot-synthesized and robot data consistently improves out-of-distribution generalization across multiple types.
  arXiv  Detail & Related papers  (2026-08-03T17:52:26Z)
• Robot Learning from Human Videos: A Survey [30.494143344658227]
  A critical bottleneck hindering further advancement in embodied AI and robotics is the challenge of scaling robot data.<n>The field of learning robot manipulation skills from human video data has attracted rapidly growing attention in recent years.<n>We present this survey to provide a comprehensive review of human-video-based learning techniques in robotics.
  arXiv  Detail & Related papers  (2026-04-30T09:11:25Z)
• EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World [63.2070298441355]
  EgoVerse is a collaborative platform for human data-driven robot learning.<n>We release 1,362 hours (80k episodes) of human demonstrations spanning 1,965 tasks, 240 scenes, and 2,087 unique demonstrators.<n>We conduct a large-scale study of human-to-robot transfer with experiments replicated across multiple labs, tasks, and robot embodiments.
  arXiv  Detail & Related papers  (2026-04-08T21:27:06Z)
• RobotSeg: A Model and Dataset for Segmenting Robots in Image and Video [56.9581053843815]
  We introduce RobotSeg, a foundation model for robot segmentation in image and video.<n>It addresses the lack of adaptation to articulated robots, reliance on manual prompts, and the need for per-frame training mask annotations.<n>It achieves state-of-the-art performance on both images and videos.
  arXiv  Detail & Related papers  (2025-11-28T07:51:02Z)
• Robot Learning with Sparsity and Scarcity [1.9203862573160633]
  I will discuss selected works on two domains: (1) tactile sensing and (2) rehabilitation robots, which are exemplars of data sparsity and scarcity, respectively.<n>My work develops machine learning algorithms that enable intent inferral with minimal data, including semi-supervised, meta-learning, and generative AI methods.
  arXiv  Detail & Related papers  (2025-09-20T23:18:41Z)
• A roadmap for AI in robotics [55.87087746398059]
  We are witnessing growing excitement in robotics at the prospect of leveraging the potential of AI to tackle some of the outstanding barriers to the full deployment of robots in our daily lives.<n>This article offers an assessment of what AI for robotics has achieved since the 1990s and proposes a short- and medium-term research roadmap listing challenges and promises.
  arXiv  Detail & Related papers  (2025-07-26T15:18:28Z)
• dARt Vinci: Egocentric Data Collection for Surgical Robot Learning at Scale [9.732963094538837]
  We introduce dARt Vinci, a scalable data collection platform for robot learning in surgical settings.<n>The system uses Augmented Reality (AR) hand tracking and a high-fidelity physics engine to capture subtle maneuvers in primitive surgical tasks.
  arXiv  Detail & Related papers  (2025-03-07T18:07:54Z)
• RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version) [25.298789781487084]
  RoboTwin is a generative digital twin framework that uses 3D generative foundation models and large language models to produce diverse expert datasets.<n>Specifically, RoboTwin creates varied digital twins of objects from single 2D images, generating realistic and interactive scenarios.<n>Our framework offers a comprehensive benchmark with both simulated and real-world data, enabling standardized evaluation and better alignment between simulated training and real-world performance.
  arXiv  Detail & Related papers  (2024-09-04T17:59:52Z)
• Semantically Controllable Augmentations for Generalizable Robot Learning [40.89398799604755]
  Generalization to unseen real-world scenarios for robot manipulation requires exposure to diverse datasets during training. We propose a generative augmentation framework for semantically controllable augmentations and rapidly multiplying robot datasets.
  arXiv  Detail & Related papers  (2024-09-02T05:25:34Z)
• NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction [19.65778558341053]
  Speech-gesture HRI datasets often focus on elementary tasks, like object pointing and pushing. We introduce NatSGD, a multimodal HRI dataset encompassing human commands through speech and gestures. We demonstrate its effectiveness in training robots to understand tasks through multimodal human commands.
  arXiv  Detail & Related papers  (2024-03-04T18:02:41Z)
• AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents [109.3804962220498]
  AutoRT is a system to scale up the deployment of operational robots in completely unseen scenarios with minimal human supervision. We demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both teleoperation and autonomous robot policies. We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
  arXiv  Detail & Related papers  (2024-01-23T18:45:54Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• SACSoN: Scalable Autonomous Control for Social Navigation [62.59274275261392]
  We develop methods for training policies for socially unobtrusive navigation. By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. We collect a large dataset where an indoor mobile robot interacts with human bystanders.
  arXiv  Detail & Related papers  (2023-06-02T19:07:52Z)
• RT-1: Robotics Transformer for Real-World Control at Scale [98.09428483862165]
  We present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.
  arXiv  Detail & Related papers  (2022-12-13T18:55:15Z)
• PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection [19.04536551595612]
  Policy Assisted TeleOperation (PATO) is a system which automates part of the demonstration collection process using a learned assistive policy. PATO autonomously executes repetitive behaviors in data collection and asks for human input only when it is uncertain about which subtask or behavior to execute.
  arXiv  Detail & Related papers  (2022-12-09T07:38:09Z)
• GNM: A General Navigation Model to Drive Any Robot [67.40225397212717]
  General goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots. We analyze the necessary design decisions for effective data sharing across robots. We deploy the trained GNM on a range of new robots, including an under quadrotor.
  arXiv  Detail & Related papers  (2022-10-07T07:26:41Z)
• Peer-Assisted Robotic Learning: A Data-Driven Collaborative Learning Approach for Cloud Robotic Systems [26.01178673629753]
  Peer-Assisted Robotic Learning (PARL) in robotics is inspired by the peer-assisted learning in cognitive psychology and pedagogy. Data and models are shared by robots to the cloud after semantic computing and training locally. Finally, fine tune this larger shared dataset in the cloud to local robots.
  arXiv  Detail & Related papers  (2020-10-16T10:52:54Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.