ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video
- URL: http://arxiv.org/abs/2607.17790v1
- Date: Mon, 20 Jul 2026 10:22:53 GMT
- Title: ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video
- Abstract summary: ReViV is a framework for holistic egocentric 4D reconstruction.<n>Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture.
- Score: 51.951128822418774
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environment. A holistic and efficient multimodal model capable of reconstructing this 4D representation is therefore highly desirable. However, existing approaches often rely on auxiliary inputs such as pre-computed camera trajectories, treat scene perception and human ego-motion modeling as separate problems despite their strong interdependency, and suffer from slow inference time. To address these limitations, we present ReViV, the first unified framework for holistic egocentric 4D reconstruction that extracts both viewer and view dynamics from a single monocular RGB video. We formulate the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body motion, hand motion, and depth. Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture to simultaneously reconstruct the temporally consistent 4D reconstruction across the viewer and the view with fast inference speed. Extensive experiments on diverse benchmarks, including HoloAssist, HOT3D, ARCTIC, Aria Digital Twin, and TACO, demonstrate that ReViV achieves state-of-the-art accuracy and efficiency across holistic ego-body, hand, and gaze reconstruction, camera tracking, while maintaining highly competitive egocentric depth estimation without relying on heavy task-specific priors. Code and models are fully open-sourced: https://reviv4d.github.io/.
Related papers
- Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videos [54.612320509347825]
We introduce Hand-4DGS, the first feed-forward framework for reconstructing dynamic 4D hands directly from egocentric videos.<n>Our approach incorporates a mesh-guided representation for structural priors and temporal convolutions to model dynamic motion.
arXiv Detail & Related papers (2026-06-17T14:58:37Z) - Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video [63.61099683012546]
We introduce RealMV-4D, a dataset of synchronized multi-view videos.<n>We then train a multi-view video diffusion model driven by a novel fused time(T)-view(V) attention mechanism.<n>Experiments demonstrate that our method outperforms existing approaches in both visual fidelity and consistency.
arXiv Detail & Related papers (2026-05-25T06:59:26Z) - Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting [3.1328424544428852]
We create a framework capable of leveraging internet-scale monocular videos.<n>Our core contribution is the generation of pseudo multi-view training triplets, consisting of a source video, a geometric anchor, and a target video.<n>At inference, our diffusion transformer utilizes a 4D point-cloud derived anchor to achieve state-of-the-art temporal consistency.
arXiv Detail & Related papers (2026-04-23T15:32:56Z) - EgoM2P: Egocentric Multimodal Multitask Pretraining [55.259234688003545]
Building large-scale egocentric multimodal and multitask models presents unique challenges.<n> EgoM2P is a masked modeling framework that learns from temporally-aware multimodal tokens to train a large, general-purpose model for egocentric 4D understanding.<n>We will fully open-source EgoM2P to support the community and advance egocentric vision research.
arXiv Detail & Related papers (2025-06-09T15:59:25Z) - Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera [49.82535393220003]
Dyn-HaMR is the first approach to reconstruct 4D global hand motion from monocular videos recorded by dynamic cameras in the wild.<n>We show that our approach significantly outperforms state-of-the-art methods in terms of 4D global mesh recovery.<n>This establishes a new benchmark for hand motion reconstruction from monocular video with moving cameras.
arXiv Detail & Related papers (2024-12-17T12:43:10Z) - Self-Supervised Monocular 4D Scene Reconstruction for Egocentric Videos [25.41337525728398]
EgoMono4D is a novel model that unifies the estimation of multiple variables necessary for Egocentric Monocular 4D reconstruction.<n>It achieves superior performance in dense pointclouds sequence reconstruction compared to all baselines.
arXiv Detail & Related papers (2024-11-14T02:57:11Z) - EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splatting [95.44545809256473]
EgoGaussian is a method capable of simultaneously reconstructing 3D scenes and dynamically tracking 3D object motion from RGB egocentric input alone.
We show significant improvements in terms of both dynamic object and background reconstruction quality compared to the state-of-the-art.
arXiv Detail & Related papers (2024-06-28T10:39:36Z) - Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer [38.85054820740242]
We present a novel approach for generating high-quality, coherent human videos from a single image.
Our framework combines the strengths of diffusion transformers for capturing global correlations and CNNs for accurate condition injection.
We demonstrate our method's ability to synthesize 360-degree realistic, coherent human motion videos.
arXiv Detail & Related papers (2024-05-27T17:53:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.