AoE: Always-on Egocentric Human Video Collection for Embodied AI

• URL: http://arxiv.org/abs/2602.23893v2
• Date: Mon, 02 Mar 2026 02:33:09 GMT
• Title: AoE: Always-on Egocentric Human Video Collection for Embodied AI
• Abstract summary: Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling.<n>We propose the Always-on Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones.
• Score: 44.083451969789216
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high infrastructure costs, complex hardware dependencies, and limited interaction scope, making scalable expansion challenging. In fact, humans themselves are ideal physically embodied agents. Therefore, obtaining egocentric real-world interaction data from globally distributed "human agents" offers advantages of low cost and sustainability. To this end, we propose the Always-on Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones, enabling low-cost, highly efficient, and scene-agnostic real-world interaction data collection to address the challenge of data scarcity. Specifically, we first employ an ergonomic neck-mounted smartphone holder to enable low-barrier, large-scale egocentric data collection through a cloud-edge collaborative architecture. Second, we develop a cross-platform mobile APP that leverages on-device compute for real-time processing, while the cloud hosts automated labeling and filtering pipelines that transform raw videos into high-quality training data. Finally, the AoE system supports distributed Ego video data collection by anyone, anytime, and anywhere. We evaluate AoE on data preprocessing quality and downstream tasks, demonstrating that high-quality egocentric data significantly boosts real-world generalization.

Related papers

• HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining [93.5192770515694]
  Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck.<n>Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment.<n>Embodied foundation models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction.
  arXiv  Detail & Related papers  (2026-06-18T17:37:34Z)
• EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks [41.23218228921582]
  EgoLive is the largest open-source annotated egocentric dataset focused on real-world task-oriented human routines to date.<n>It delivers leading data quality via a customized head-mounted capture device and comprehensive high-precision multi-modal annotations.<n>All data is collected exclusively in unconstrained real-world scenarios and encompasses vertical field human working data, including home service, retail, and other practical work scenarios.
  arXiv  Detail & Related papers  (2026-04-26T07:21:15Z)
• Gypscie: A Cross-Platform AI Artifact Management System [0.6784745592354215]
  We introduce Gypscie, a cross-platform AI artifact management system.<n>By providing a unified view of all AI artifacts, Gypscie simplifies the development and deployment of AI applications.
  arXiv  Detail & Related papers  (2026-04-11T18:24:26Z)
• $Ψ_0$: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation [39.811210435945924]
  We introduce an open foundation model to address challenging humanoid loco-manipulation tasks.<n>Our research identifies a critical yet often overlooked data recipe.<n>We demonstrate that pre-training on high-quality egocentric human manipulation data followed by post-training on domain-specific real-world humanoid trajectories yields superior performance.
  arXiv  Detail & Related papers  (2026-03-12T17:59:51Z)
• Percepta: High Performance Stream Processing at the Edge [0.0]
  This paper presents Percepta, a lightweight Data Stream Processing (DSP) system tailored to support AI workloads at the edge.<n>Additional functionalities include data normalization, harmonization across heterogeneous protocols and sampling rates, and robust handling of missing or incomplete data.
  arXiv  Detail & Related papers  (2025-10-02T08:57:45Z)
• Developing Vision-Language-Action Model from Egocentric Videos [14.1517430035289]
  Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation.<n>Prior studies that leverage such videos for training robot policies typically rely on auxiliary annotations, such as detailed hand-pose recordings.<n>In this work, we address this challenge by leveraging EgoScaler, a framework that extracts 6DoF object manipulation trajectories from egocentric videos.
  arXiv  Detail & Related papers  (2025-09-26T07:09:33Z)
• InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation [54.09384502044162]
  We introduce InterAct, a large-scale 3D HOI benchmark featuring dataset and methodological advancements.<n>First, we consolidate and standardize 21.81 hours of HOI data from diverse sources, enriching it with detailed textual annotations.<n>Second, we propose a unified optimization framework to enhance data quality by reducing artifacts and correcting hand motions.<n>Third, we define six benchmarking tasks and develop a unified HOI generative modeling perspective, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-11T15:43:54Z)
• AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems [88.05152114775498]
  AgiBot World is a large-scale platform comprising over 1 million trajectories across 217 tasks in five deployment scenarios.<n>AgiBot World guarantees high-quality and diverse data distribution.<n>GO-1 exhibits exceptional capability in real-world dexterous and long-horizon tasks.
  arXiv  Detail & Related papers  (2025-03-09T15:40:29Z)
• When Incentives Backfire, Data Stops Being Human [27.26816810619047]
  We argue that the widespread use of large language models threatens the quality and integrity of human-generated data.<n>Existing data collection systems prioritize speed, scale, and efficiency at the cost of intrinsic human motivation.<n>We propose rethinking data collection systems to align with contributors' intrinsic motivations.
  arXiv  Detail & Related papers  (2025-02-11T17:51:52Z)
• MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Users [52.696186533146516]
  MobileA3gent is a collaborative framework that trains mobile GUI Agents using decentralized self-sourced data.<n>MobileA3gent achieves superior performance over traditional approaches at only 1% of the cost.
  arXiv  Detail & Related papers  (2025-02-05T08:26:17Z)
• OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis [55.390060529534644]
  We propose OS-Genesis, a novel data synthesis pipeline for Graphical User Interface (GUI) agents.<n>Instead of relying on pre-defined tasks, OS-Genesis enables agents first to perceive environments and perform step-wise interactions.<n>We demonstrate that training GUI agents with OS-Genesis significantly improves their performance on highly challenging online benchmarks.
  arXiv  Detail & Related papers  (2024-12-27T16:21:58Z)
• Data-centric Artificial Intelligence: A Survey [47.24049907785989]
  Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
  arXiv  Detail & Related papers  (2023-03-17T17:44:56Z)
• Data-centric AI: Perspectives and Challenges [51.70828802140165]
  Data-centric AI (DCAI) advocates a fundamental shift from model advancements to ensuring data quality and reliability. We bring together three general missions: training data development, inference data development, and data maintenance.
  arXiv  Detail & Related papers  (2023-01-12T05:28:59Z)
• Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy [2.9005223064604078]
  We introduce a scalable Aerial Synthetic Data Augmentation (ASDA) framework tailored to aerial autonomy applications. ASDA extends a central data collection engine with two scriptable pipelines that automatically perform scene and data augmentations. We demonstrate the effectiveness of our method in automatically generating diverse datasets.
  arXiv  Detail & Related papers  (2022-11-10T04:37:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.