EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

• URL: http://arxiv.org/abs/2605.19004v1
• Date: Mon, 18 May 2026 18:26:51 GMT
• Title: EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction
• Abstract summary: EgoTraj is an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro)<n>It contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments.<n>To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation.
• Score: 4.1965237414146666
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensing systems, and assistive navigation. However, progress in this direction remains limited due to the scarcity of egocentric trajectory datasets collected in real-world environments. Addressing this need, we introduce EgoTraj, an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro). EgoTraj contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments. Each recording provides synchronized RGB video along with ground-truth data, including continuous time-synchronized 6-degree-of-freedom head poses, per-frame 3D eye gaze vectors, scene annotations. To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation across diverse urban routes with broad participant diversity. To demonstrate the potential of the dataset, we benchmark several state-of-the-art methods for egocentric trajectory prediction and conduct ablation studies to analyze the contributions of gaze, scene, and motion cues. The results highlight the utility of EgoTraj for AR-based perception, navigation, and assistive systems. The EgoTraj dataset, code, and EgoViz Dashboard are publicly available at https://github.com/yehiahmad/EgoTraj.

Related papers

• EgoSim: Egocentric World Simulator for Embodied Interaction Generation [93.11209644808783]
  EgoSim is a closed-loop egocentric world simulator that generates spatially consistent interaction videos.<n>It persistently updates the underlying 3D scene state for continuous simulation.<n>EgoSim significantly outperforms existing methods in terms of visual quality, spatial consistency, and generalization.
  arXiv  Detail & Related papers  (2026-04-01T15:00:46Z)
• ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios [50.47300468137957]
  ENIGMA-360 is a new ego-exo dataset acquired in a real industrial scenario.<n>The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized.<n>These results highlight the need for new models capable of robust ego-exo understanding in real-world environments.
  arXiv  Detail & Related papers  (2026-03-10T14:47:15Z)
• LookOut: Real-World Humanoid Egocentric Navigation [61.14016011125957]
  We introduce the challenging problem of predicting a sequence of future 6D head poses from an egocentric video.<n>To solve this task, we propose a framework that reasons over temporally aggregated 3D latent features.<n>Motivated by the lack of training data in this space, we present a dataset collected through this approach.
  arXiv  Detail & Related papers  (2025-08-20T06:43:36Z)
• EgoMe: A New Dataset and Challenge for Following Me via Egocentric View in Real World [12.699670048897085]
  In human imitation learning, the imitator typically take the egocentric view as a benchmark, naturally transferring behaviors observed from an exocentric view to their owns.<n>We introduce EgoMe, which towards following the process of human imitation learning via the imitator's egocentric view in the real world.<n>Our dataset includes 7902 paired exo-ego videos spanning diverse daily behaviors in various real-world scenarios.
  arXiv  Detail & Related papers  (2025-01-31T11:48:22Z)
• EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World [44.34800426136217]
  We introduce EgoExoLearn, a dataset that emulates the human demonstration following process.<n>EgoExoLearn contains egocentric and demonstration video data spanning 120 hours.<n>We present benchmarks such as cross-view association, cross-view action planning, and cross-view referenced skill assessment.
  arXiv  Detail & Related papers  (2024-03-24T15:00:44Z)
• EgoGen: An Egocentric Synthetic Data Generator [53.32942235801499]
  EgoGen is a new synthetic data generator that can produce accurate and rich ground-truth training data for egocentric perception tasks. At the heart of EgoGen is a novel human motion synthesis model that directly leverages egocentric visual inputs of a virtual human to sense the 3D environment. We demonstrate EgoGen's efficacy in three tasks: mapping and localization for head-mounted cameras, egocentric camera tracking, and human mesh recovery from egocentric views.
  arXiv  Detail & Related papers  (2024-01-16T18:55:22Z)
• EgoHumans: An Egocentric 3D Multi-Human Benchmark [37.375846688453514]
  We present EgoHumans, a new multi-view multi-human video benchmark to advance the state-of-the-art of egocentric human 3D pose estimation and tracking. We propose a novel 3D capture setup to construct a comprehensive egocentric multi-human benchmark in the wild. We leverage consumer-grade wearable camera-equipped glasses for the egocentric view, which enables us to capture dynamic activities like playing tennis, fencing, volleyball, etc.
  arXiv  Detail & Related papers  (2023-05-25T21:37:36Z)
• EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset [19.496721051685135]
  Embodied tracking is a key component to many egocentric vision problems. EgoTracks is a new dataset for long-term egocentric visual object tracking. We show improvements that can be made to a STARK tracker to significantly increase its performance on egocentric data.
  arXiv  Detail & Related papers  (2023-01-09T09:10:35Z)
• Ego-Body Pose Estimation via Ego-Head Pose Estimation [22.08240141115053]
  Estimating 3D human motion from an egocentric video sequence plays a critical role in human behavior understanding and has various applications in VR/AR. We propose a new method, Ego-Body Pose Estimation via Ego-Head Pose Estimation (EgoEgo), which decomposes the problem into two stages, connected by the head motion as an intermediate representation. This disentanglement of head and body pose eliminates the need for training datasets with paired egocentric videos and 3D human motion.
  arXiv  Detail & Related papers  (2022-12-09T02:25:20Z)
• Egocentric Video-Language Pretraining [74.04740069230692]
  Video-Language Pretraining aims to learn transferable representation to advance a wide range of video-text downstream tasks. We exploit the recently released Ego4D dataset to pioneer Egocentric training along three directions. We demonstrate strong performance on five egocentric downstream tasks across three datasets.
  arXiv  Detail & Related papers  (2022-06-03T16:28:58Z)
• Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos [92.38049744463149]
  We introduce an approach for pre-training egocentric video models using large-scale third-person video datasets. Our idea is to discover latent signals in third-person video that are predictive of key egocentric-specific properties. Our experiments show that our Ego-Exo framework can be seamlessly integrated into standard video models.
  arXiv  Detail & Related papers  (2021-04-16T06:10:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.