EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

• URL: http://arxiv.org/abs/2603.00167v1
• Date: Thu, 26 Feb 2026 09:56:21 GMT
• Title: EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations
• Abstract summary: Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning.<n>We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation.<n>Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision.
• Score: 3.5831173106587393
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Efficient navigation in dynamic environments requires anticipating how motion patterns evolve beyond the robot's immediate perceptual range, enabling preemptive rather than purely reactive planning in crowded scenes. Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning, but constructing them traditionally requires global environment observations over extended periods of time. We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation. Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision, allowing local observations to serve as predictive signals of global motion structure. Thanks to this, we offer the capacity to forecast future motion dynamics over the whole environment rather than merely extend past patterns in the robot's field of view. Experiments in large simulated environments show that EgoMoD accurately predicts future MoDs under limited observability, while evaluation with real images showcases its zero-shot transferability to real systems.

Related papers

• EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan [13.267842001048502]
  This paper proposes a transferable Map of Dynamics (MoD) framework that generalizes to unknown environments.<n>MoDs are essential for encoding human motion characteristics to enable accurate pedestrian trajectory prediction or safe robot navigation.<n>Our framework trains a continuous, LiDAR-based MoD estimator across diverse environments.
  arXiv  Detail & Related papers  (2026-09-05T17:35:21Z)
• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• DynaMOMA: Instantaneous Prediction of Grasp Poses for Mobile Manipulation of Dynamic Objects [27.42982156996797]
  We propose a dynamic mobile manipulation framework that couples instantaneous grasp trajectory prediction with whole-body control policy.<n>Our method achieves strong performance in mobile manipulation of dynamic objects across diverse settings and grasping metrics.
  arXiv  Detail & Related papers  (2026-06-24T01:58:15Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning [13.877624821024535]
  We propose a non-autoregressive occupancy world model for future semantic occupancy forecasting and motion planning.<n>We show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance.
  arXiv  Detail & Related papers  (2026-05-17T22:52:53Z)
• ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving [40.28153843744977]
  We propose Temporal Residual World Model (TR-World), which focuses on dynamic object modeling.<n>By calculating the temporal residuals of scene representations, the information of dynamic objects can be extracted without relying on detection and tracking.<n>We also propose Future-Guided Trajectory Refinement (FGTR) module, which conducts interaction between prior trajectories and the future BEV features.
  arXiv  Detail & Related papers  (2026-02-11T14:12:26Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• Walk through Paintings: Egocentric World Models from Internet Priors [65.30611174953958]
  We present the Egocentric World Model (EgoWM), a simple, architecture-agnostic method that transforms any pretrained video diffusion model into an action-conditioned world model.<n>Rather than training from scratch, we repurpose the rich world priors of Internet-scale video models and inject motor commands through lightweight conditioning layers.<n>Our approach scales naturally across embodiments and action spaces, ranging from 3-DoF mobile robots to 25-DoF humanoids.
  arXiv  Detail & Related papers  (2026-01-21T18:59:32Z)
• Back to the Features: DINO as a Foundation for Video World Models [29.116268020812072]
  We present DINO-world, a powerful generalist video world model trained to predict future frames in the latent space of DINOv2.<n>By leveraging a pre-trained image encoder and training a future predictor on a large-scale uncurated video dataset, DINO-world learns the temporal dynamics of diverse scenes.
  arXiv  Detail & Related papers  (2025-07-25T17:54:10Z)
• Dfferentiable Raycasting for Self-supervised Occupancy Forecasting [52.61762537741392]
  Motion planning for autonomous driving requires learning how the environment around an ego-vehicle evolves with time. In this paper, we use geometric occupancy as a natural alternative to view-dependent representations such as freespace. Our key insight is to use differentiable raycasting to "render" future occupancy predictions into future LiDAR sweep predictions.
  arXiv  Detail & Related papers  (2022-10-04T21:35:21Z)
• Motion Transformer with Global Intention Localization and Local Movement Refinement [103.75625476231401]
  Motion TRansformer (MTR) models motion prediction as the joint optimization of global intention localization and local movement refinement. MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges.
  arXiv  Detail & Related papers  (2022-09-27T16:23:14Z)
• Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning [63.25627328308978]
  We propose a-temporal prediction network pipeline to generate future occupancy predictions. Compared to current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds. We publicly release our grid occupancy dataset based on nulis to support further research.
  arXiv  Detail & Related papers  (2022-05-06T13:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.