ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine

• URL: http://arxiv.org/abs/2607.28625v1
• Date: Thu, 30 Jul 2026 17:59:50 GMT
• Title: ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine
• Abstract summary: Ambient Capture Engine (ACE) transforms real home environments into spatially, temporally synchronized recording studios.<n>ACE records egocentric and multi-view exocentric video, full-body and articulated hand motion, object geometry and 6-DoF trajectories, audio, and signals as a unified multisensory stream.<n>Dataset spans atomic manipulation, long-horizon chains of household activities, and human-scene interaction, while preserving natural behavioral variation through goal-level rather than step-by-step instructions.
• Score: 83.02804728003862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied intelligence faces a fundamental data bottleneck. Models must capture how first-person perception, whole-body motion, dexterous manipulation, object state, sound, and touch evolve together as humans pursue goals over time. Existing datasets fragment this experience across viewpoints, modalities, or spatial scales, leaving the full perception-action loop only partially observed. We introduce the Ambient Capture Engine (ACE), a human-centric data engine that transforms real home environments into spatially calibrated, temporally synchronized recording studios. ACE operates at two complementary scales: a table-scale configuration resolves hand-object manipulation, while a room-scale configuration captures whole-body motion, locomotion, and interactions across a furnished home. ACE records egocentric and multi-view exocentric video, full-body and articulated hand motion, object geometry and 6-DoF trajectories, audio, and tactile signals as a unified multisensory stream. Using ACE, we build ACE-Data-0, comprising 150 hours and 17M video frames across 200 task categories, performed by 50 participants in 2 environments, for a total of 75,000 interaction episodes. The dataset spans atomic manipulation, long-horizon chains of household activities, and human-scene interaction, while preserving natural behavioral variation through goal-level rather than step-by-step instructions. We further introduce a hierarchical benchmark that progresses from signals to scene components and then to interactions. Evaluations of state-of-the-art methods expose substantial gaps under contact, occlusion, egomotion, and long temporal horizons. ACE-Data-0 provides synchronized human demonstrations with aligned perceptual, kinematic, and contact supervision, offering a scalable foundation for imitation learning, world models, vision-language-action systems, and embodied AI.

Related papers

• HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction [48.4618478154232]
  HiPHI is a 600+ hour scale high-fidelity whole-body human motion dataset.<n>It provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories.<n>We introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications.
  arXiv  Detail & Related papers  (2026-08-17T07:54:26Z)
• VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes [71.14444761145435]
  Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion.<n>No existing data source provides this complete at scale.<n>We address this bottleneck by generating vision-language-kinematics supervision synthetically in reconstructed scenes.
  arXiv  Detail & Related papers  (2026-06-29T17:59:55Z)
• ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining [40.678451625177026]
  Vision-Language-Action (VLA) models benefit from large-scale and diverse embodied data, yet scaling robot trajectory collection is costly and labor-intensive.<n>Recent advances show that large-scale egocentric human videos provide complementary real-world supervision in pretraining.<n>We introduce ACE-EGO-0, a unified VLA pretraining framework jointly leveraging heterogeneous data sources.
  arXiv  Detail & Related papers  (2026-06-15T18:40:18Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• Revisit Human-Scene Interaction via Space Occupancy [55.67657438543008]
  Human-scene Interaction (HSI) generation is a challenging task and crucial for various downstream tasks. In this work, we argue that interaction with a scene is essentially interacting with the space occupancy of the scene from an abstract physical perspective. By treating pure motion sequences as records of humans interacting with invisible scene occupancy, we can aggregate motion-only data into a large-scale paired human-occupancy interaction database.
  arXiv  Detail & Related papers  (2023-12-05T12:03:00Z)
• Object Motion Guided Human Motion Synthesis [22.08240141115053]
  We study the problem of full-body human motion synthesis for the manipulation of large-sized objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework. We develop a novel system that captures full-body human manipulation motions by simply attaching a smartphone to the object being manipulated.
  arXiv  Detail & Related papers  (2023-09-28T08:22:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.