EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation

• URL: http://arxiv.org/abs/2604.01421v1
• Date: Wed, 01 Apr 2026 21:43:57 GMT
• Title: EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation
• Abstract summary: We present EgoFlow, a flow-matching framework that synthesizes realistic and physically plausible trajectories conditioned on multimodal egocentric observations.<n>Our results highlight the promise of flow-based generative modeling for scalable and physically grounded egocentric motion understanding.
• Score: 47.32597153743819
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding and predicting object motion from egocentric video is fundamental to embodied perception and interaction. However, generating physically consistent 6DoF trajectories remains challenging due to occlusions, fast motion, and the lack of explicit physical reasoning in existing generative models. We present EgoFlow, a flow-matching framework that synthesizes realistic and physically plausible trajectories conditioned on multimodal egocentric observations. EgoFlow employs a hybrid Mamba-Transformer-Perceiver architecture to jointly model temporal dynamics, scene geometry, and semantic intent, while a gradient-guided inference process enforces differentiable physical constraints such as collision avoidance and motion smoothness. This combination yields coherent and controllable motion generation without post-hoc filtering or additional supervision. Experiments on real-world datasets HD-EPIC, EgoExo4D, and HOT3D show that EgoFlow outperforms diffusion-based and transformer baselines in accuracy, generalization, and physical realism, reducing collision rates by up to 79%, and strong generalization to unseen scenes. Our results highlight the promise of flow-based generative modeling for scalable and physically grounded egocentric motion understanding.

Related papers

• EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generation [74.07852250099559]
  We investigate Egocentric Vision-Language (Ego-VL) motion generation.<n>This task requires synthesizing 3D human motion conditioned jointly on first-person visual observations and natural language instructions.<n>We propose a hierarchical generative framework textbfEgoMotion to address this challenge.
  arXiv  Detail & Related papers  (2026-04-21T05:31:06Z)
• HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching [113.81911881001905]
  HO-Flow is a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects.<n>It first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold.<n>It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation.
  arXiv  Detail & Related papers  (2026-04-12T22:06:11Z)
• EgoSim: Egocentric World Simulator for Embodied Interaction Generation [93.11209644808783]
  EgoSim is a closed-loop egocentric world simulator that generates spatially consistent interaction videos.<n>It persistently updates the underlying 3D scene state for continuous simulation.<n>EgoSim significantly outperforms existing methods in terms of visual quality, spatial consistency, and generalization.
  arXiv  Detail & Related papers  (2026-04-01T15:00:46Z)
• EgoForge: Goal-Directed Egocentric World Simulator [26.712565464146937]
  EgoForge is a goal-directed world simulator that generates coherent, first-person video rollouts from minimal static inputs.<n>VideoDiffusionNFT is a trajectory-level reward-guided refinement that optimize goal completion, temporal causality, scene consistency, and perceptual fidelity during diffusion sampling.
  arXiv  Detail & Related papers  (2026-03-20T17:46:55Z)
• Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis [25.153233500074194]
  EgoHOI is an egocentric HOI world model that breaks away from this shortcut to simulate photorealistic, contact-consistent interactions from action signals alone.<n>EgoHOI distills geometric and kinematic priors from 3D estimates into physics-informed embeddings.<n>Experiments on the HOT3D dataset demonstrate consistent gains over strong baselines, and ablations validate the effectiveness of our design.
  arXiv  Detail & Related papers  (2026-03-13T21:46:17Z)
• EgoReAct: Egocentric Video-Driven 3D Human Reaction Generation [84.37917777533963]
  We present EgoReAct, the first framework that generates 3D-aligned human reaction motions from egocentric video streams in real-time.<n>EgoReAct achieves remarkably higher realism, spatial consistency, and generation efficiency compared with prior methods.
  arXiv  Detail & Related papers  (2025-12-28T06:44:05Z)
• SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios [48.09735396455107]
  Hand-Object Interaction (HOI) generation has significant application potential.<n>Current 3D HOI motion generation approaches heavily rely on predefined 3D object models and lab-captured motion data.<n>We propose a novel framework that combines visual priors and dynamic constraints within a synchronized diffusion process to generate the HOI video and motion simultaneously.
  arXiv  Detail & Related papers  (2025-06-03T05:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.