GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth

• URL: http://arxiv.org/abs/2605.10525v3
• Date: Wed, 13 May 2026 02:23:53 GMT
• Title: GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth
• Abstract summary: Video depth estimation extends monocular prediction into the temporal domain to ensure coherence.<n>Current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency.<n>We propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency.
• Score: 12.866152238833104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video depth estimation extends monocular prediction into the temporal domain to ensure coherence. However, existing methods often suffer from spatial blurring in fine-detail regions and temporal inconsistencies. We argue that current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency-particularly under rotations or drastic view changes. To address this, we propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency. Distinctively, GemDepth introduces a Geometry-Embedding Module (GEM) that predicts inter-frame camera poses to generate implicit geometric embeddings. This injection of motion priors equips the network with intrinsic 3D perception and alignment capabilities. Guided by these geometric cues, our Alternating Spatio-Temporal Transformer (ASTT) captures latent point-level correspondences to simultaneously enhance spatial precision for sharp details and enforce rigorous temporal consistency. Furthermore, GemDepth employs a data-efficient training strategy, effectively bridging the gap between high efficiency and robust geometric consistency. As shown in Fig.2, comprehensive evaluations demonstrate that GemDepth achieves state-of-the-art performance across multiple datasets, particularly in complex dynamic scenarios. The code is publicly available at: https://github.com/Yuecheng919/GemDepth.

Related papers

• Geometry-Grounded Unified 3D Perception for Autonomous Driving [36.11730832045922]
  Camera-based autonomous driving perception requires a shared representation that preserves metric 3D structure across synchronized multi-camera streams.<n>We present a Geometry-grounded Unified 3D Perception framework that adapts the reconstruction-oriented latent of VGGT to calibrated, streaming driving scenes.
  arXiv  Detail & Related papers  (2026-08-13T12:17:50Z)
• IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer [64.5765465030666]
  We propose IGGT4D, a streaming instance-grounded geometry Transformer for online 4D scene understanding.<n> IGGT4D processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally updates a unified representation of camera motion, geometry, and object identity.<n> Experiments on 3D reconstruction, pose estimation, instance spatial tracking, and open-vocabulary segmentation demonstrate that IGGT4D outperforms existing streaming baselines.
  arXiv  Detail & Related papers  (2026-07-21T16:00:01Z)
• Reliability-Aware 3D Geometric Injection for Universal Person Re-identification [47.874819213762926]
  We propose a Universal Monocular 3D-Enhanced ReID framework driven by a Consistency-Aware Reliability Gate and Dual-Stream Residual Fusion.<n>Our method improves challenging, structure-sensitive scenarios while preserving competitive performance on clean domains.
  arXiv  Detail & Related papers  (2026-07-21T08:54:25Z)
• GeoRect4D: Geometry-Compatible Generative Rectification for Dynamic Sparse-View 3D Reconstruction [16.448484686109992]
  Reconstructing dynamic 3D scenes from multi-view videos is highly ill-posed, often leading to geometric collapse, trajectory drift, and floating artifacts.<n>Recent attempts introduce generative priors to hallucinate missing content, yet naive integration frequently causes structural drift and temporal inconsistency.<n>We propose GeoRect4D, a novel unified framework for sparse-view dynamic reconstruction that couples explicit 3D consistency with generative refinement.
  arXiv  Detail & Related papers  (2026-04-22T17:12:02Z)
• Geometry-Aware Rotary Position Embedding for Consistent Video World Model [48.914346802616414]
  ViewRope is a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers.<n>Geometry-Aware Frame-Sparse Attention exploits these geometric cues to selectively attend to relevant historical frames.<n>Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.
  arXiv  Detail & Related papers  (2026-02-08T08:01:16Z)
• GeoVideo: Introducing Geometric Regularization into Video Generation Model [46.38507581500745]
  We introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction.<n>Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved structural coherence-temporal shape, consistency, and physical plausibility.
  arXiv  Detail & Related papers  (2025-12-03T05:11:57Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstruction [53.19968902152528]
  We present POMATO, a unified framework for dynamic 3D reconstruction by marrying pointmap matching with temporal motion.<n>Specifically, our method learns an explicit matching relationship by mapping RGB pixels from both dynamic and static regions across different views to 3D pointmaps.<n>We show the effectiveness of the proposed pointmap matching and temporal fusion paradigm by demonstrating the remarkable performance across multiple downstream tasks.
  arXiv  Detail & Related papers  (2025-04-08T05:33:13Z)
• Graph and Skipped Transformer: Exploiting Spatial and Temporal Modeling Capacities for Efficient 3D Human Pose Estimation [36.93661496405653]
  We take a global approach to exploit Transformer-temporal information with a concise Graph and Skipped Transformer architecture. Specifically, in 3D pose stage, coarse-grained body parts are deployed to construct a fully data-driven adaptive model. Experiments are conducted on Human3.6M, MPI-INF-3DHP and Human-Eva benchmarks.
  arXiv  Detail & Related papers  (2024-07-03T10:42:09Z)
• Dynamic 3D Point Cloud Sequences as 2D Videos [81.46246338686478]
  3D point cloud sequences serve as one of the most common and practical representation modalities of real-world environments. We propose a novel generic representation called textitStructured Point Cloud Videos (SPCVs) SPCVs re-organizes a point cloud sequence as a 2D video with spatial smoothness and temporal consistency, where the pixel values correspond to the 3D coordinates of points.
  arXiv  Detail & Related papers  (2024-03-02T08:18:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.