ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios

• URL: http://arxiv.org/abs/2603.09741v2
• Date: Wed, 11 Mar 2026 14:52:42 GMT
• Title: ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios
• Abstract summary: ENIGMA-360 is a new ego-exo dataset acquired in a real industrial scenario.<n>The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized.<n>These results highlight the need for new models capable of robust ego-exo understanding in real-world environments.
• Score: 50.47300468137957
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding human behavior from complementary egocentric (ego) and exocentric (exo) points of view enables the development of systems that can support workers in industrial environments and enhance their safety. However, progress in this area is hindered by the lack of datasets capturing both views in realistic industrial scenarios. To address this gap, we propose ENIGMA-360, a new ego-exo dataset acquired in a real industrial scenario. The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized offering complementary information of the same scene. The 360 videos have been labeled with temporal and spatial annotations, enabling the study of different aspects of human behavior in industrial domain. We provide baseline experiments for 3 foundational tasks for human behavior understanding: 1) Temporal Action Segmentation, 2) Keystep Recognition and 3) Egocentric Human-Object Interaction Detection, showing the limits of state-of-the-art approaches on this challenging scenario. These results highlight the need for new models capable of robust ego-exo understanding in real-world environments. We publicly release the dataset and its annotations at https://fpv-iplab.github.io/ENIGMA-360/.

Related papers

• ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine [83.02804728003862]
  Ambient Capture Engine (ACE) transforms real home environments into spatially, temporally synchronized recording studios.<n>ACE records egocentric and multi-view exocentric video, full-body and articulated hand motion, object geometry and 6-DoF trajectories, audio, and signals as a unified multisensory stream.<n>Dataset spans atomic manipulation, long-horizon chains of household activities, and human-scene interaction, while preserving natural behavioral variation through goal-level rather than step-by-step instructions.
  arXiv  Detail & Related papers  (2026-07-30T17:59:50Z)
• EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction [4.1965237414146666]
  EgoTraj is an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro)<n>It contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments.<n>To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation.
  arXiv  Detail & Related papers  (2026-05-18T18:26:51Z)
• EgoSim: Egocentric World Simulator for Embodied Interaction Generation [93.11209644808783]
  EgoSim is a closed-loop egocentric world simulator that generates spatially consistent interaction videos.<n>It persistently updates the underlying 3D scene state for continuous simulation.<n>EgoSim significantly outperforms existing methods in terms of visual quality, spatial consistency, and generalization.
  arXiv  Detail & Related papers  (2026-04-01T15:00:46Z)
• Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions [110.43343503158306]
  This paper embeds the manual-assisted task into a vision-language-action framework, where the assistant provides services to the instructor following egocentric vision and commands.<n>Under this setting, we accomplish InterVLA, the first large-scale human-object-human interaction dataset with 11.4 hours and 1.2M frames of multimodal data.<n>We establish novel benchmarks on egocentric human motion estimation, interaction synthesis, and interaction prediction with comprehensive analysis.
  arXiv  Detail & Related papers  (2025-08-06T17:46:23Z)
• EgoMe: A New Dataset and Challenge for Following Me via Egocentric View in Real World [12.699670048897085]
  In human imitation learning, the imitator typically take the egocentric view as a benchmark, naturally transferring behaviors observed from an exocentric view to their owns.<n>We introduce EgoMe, which towards following the process of human imitation learning via the imitator's egocentric view in the real world.<n>Our dataset includes 7902 paired exo-ego videos spanning diverse daily behaviors in various real-world scenarios.
  arXiv  Detail & Related papers  (2025-01-31T11:48:22Z)
• Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning [80.37314291927889]
  We present EMBED, a method designed to transform exocentric video-language data for egocentric video representation learning. Egocentric videos predominantly feature close-up hand-object interactions, whereas exocentric videos offer a broader perspective on human activities. By applying both vision and language style transfer, our framework creates a new egocentric dataset.
  arXiv  Detail & Related papers  (2024-08-07T06:10:45Z)
• EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World [44.34800426136217]
  We introduce EgoExoLearn, a dataset that emulates the human demonstration following process.<n>EgoExoLearn contains egocentric and demonstration video data spanning 120 hours.<n>We present benchmarks such as cross-view association, cross-view action planning, and cross-view referenced skill assessment.
  arXiv  Detail & Related papers  (2024-03-24T15:00:44Z)
• EgoGen: An Egocentric Synthetic Data Generator [53.32942235801499]
  EgoGen is a new synthetic data generator that can produce accurate and rich ground-truth training data for egocentric perception tasks. At the heart of EgoGen is a novel human motion synthesis model that directly leverages egocentric visual inputs of a virtual human to sense the 3D environment. We demonstrate EgoGen's efficacy in three tasks: mapping and localization for head-mounted cameras, egocentric camera tracking, and human mesh recovery from egocentric views.
  arXiv  Detail & Related papers  (2024-01-16T18:55:22Z)
• MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain [23.598727613908853]
  We present MECCANO, a dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view.
  arXiv  Detail & Related papers  (2022-09-19T00:52:42Z)
• Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos [92.38049744463149]
  We introduce an approach for pre-training egocentric video models using large-scale third-person video datasets. Our idea is to discover latent signals in third-person video that are predictive of key egocentric-specific properties. Our experiments show that our Ego-Exo framework can be seamlessly integrated into standard video models.
  arXiv  Detail & Related papers  (2021-04-16T06:10:10Z)
• The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain [20.99718135562034]
  We introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios.
  arXiv  Detail & Related papers  (2020-10-12T12:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.