HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation

• URL: http://arxiv.org/abs/2606.31682v1
• Date: Tue, 30 Jun 2026 13:58:19 GMT
• Title: HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation
• Abstract summary: HABIT is a large-scale robot dataset demonstration for human-present environments.<n>We organize tasks into three roles capturing distinct modes of human-robot interaction: Collaborator, Coworker, and Supervisor.<n>Our experiments show that training on human-present data elicits human-aware behaviors that robot-only data fails to produce.
• Score: 46.45333030782776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale demonstration datasets have been central to recent progress in general-purpose robot policies. However, existing datasets are collected in human-absent settings, and policies trained on such data may perform tasks competently in isolation but fail to exhibit human-aware behaviors. To address this gap, we introduce HABIT, a large-scale robot demonstration dataset for human-present environments. We organize tasks into three roles capturing distinct modes of human-robot interaction: Collaborator, where human and robot jointly accomplish a task; Coworker, where they pursue separate tasks in a shared space; and Supervisor, where the human directs the robot. The dataset comprises over 10K episodes and over 160 hours across 60 tasks. Our experiments show that training on human-present data elicits human-aware behaviors that robot-only data fails to produce: spatiotemporal synchronization in Collaborator tasks, yielding in Coworker tasks, and gesture grounding in Supervisor tasks. Moreover, training on HABIT enables rapid adaptation to new human-robot interaction tasks. By introducing human presence as a new axis of dataset diversity, HABIT extends robot policies to environments shared with humans.

Related papers

• HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos [58.9564236347451]
  HumanEgo is a framework that bridges the embodiment gap between human and robot.<n>It lifts each human demonstration to an entity-level representation of hand-object interaction.<n>HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable.
  arXiv  Detail & Related papers  (2026-05-24T08:26:41Z)
• EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World [63.2070298441355]
  EgoVerse is a collaborative platform for human data-driven robot learning.<n>We release 1,362 hours (80k episodes) of human demonstrations spanning 1,965 tasks, 240 scenes, and 2,087 unique demonstrators.<n>We conduct a large-scale study of human-to-robot transfer with experiments replicated across multiple labs, tasks, and robot embodiments.
  arXiv  Detail & Related papers  (2026-04-08T21:27:06Z)
• HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos [74.43500240121476]
  We present HumanX, a full-stack framework that compiles human video into generalizable, real-world interaction skills for humanoids.<n>HumanX achieves over 8 times higher generalization success than prior methods.
  arXiv  Detail & Related papers  (2026-02-02T18:53:01Z)
• Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration [28.96294944351617]
  This review identifies and connects key components enabling intuitive information exchange and skill transfer between humans and robots.<n>We examine the full interaction pipeline: from the human-to-robot communication bridge translating multimodal inputs into robot-understandable representations.
  arXiv  Detail & Related papers  (2025-11-11T19:49:30Z)
• Humanoid Policy ~ Human Policy [41.34186233320398]
  We train a human-humanoid behavior policy, which we term Human Action Transformer (HAT)<n>The state-action space of HAT is unified for both humans and humanoid robots and can be differentiably retargeted to robot actions.<n>We show that human data improves both generalization and robustness of HAT with significantly better data collection efficiency.
  arXiv  Detail & Related papers  (2025-03-17T17:59:09Z)
• InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions [7.574421886354134]
  InteRACT architecture pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines.
  arXiv  Detail & Related papers  (2023-11-21T19:15:17Z)
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots [119.55240471433302]
  Habitat 3.0 is a simulation platform for studying collaborative human-robot tasks in home environments. It addresses challenges in modeling complex deformable bodies and diversity in appearance and motion. Human-in-the-loop infrastructure enables real human interaction with simulated robots via mouse/keyboard or a VR interface.
  arXiv  Detail & Related papers  (2023-10-19T17:29:17Z)
• Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration [83.4081612443128]
  We show that a short calibration using REMP can effectively bridge the gap between what a non-expert user thinks a robot can reach and the ground-truth. We show that this calibration procedure not only results in better user perception, but also promotes more efficient human-robot collaborations.
  arXiv  Detail & Related papers  (2021-03-06T09:14:30Z)
• Human-robot co-manipulation of extended objects: Data-driven models and control from analysis of human-human dyads [2.7036498789349244]
  We use data from human-human dyad experiments to determine motion intent which we use for a physical human-robot co-manipulation task. We develop a deep neural network based on motion data from human-human trials to predict human intent based on past motion.
  arXiv  Detail & Related papers  (2020-01-03T21:23:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.