Video Generative Models as Geometry Learner

• URL: http://arxiv.org/abs/2608.28549v1
• Date: Fri, 28 Aug 2026 17:25:31 GMT
• Title: Video Generative Models as Geometry Learner
• Abstract summary: Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation.<n>We repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation.<n>Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the video model, while further adapting them for joint modeling of images and geometry targets.
• Score: 88.78682297429937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these geometric targets, or (ii) jointly fine-tune modified image diffusion backbones (e.g., altered self-attention), which typically demands substantial labeled data. To overcome these limitations in a principled fashion, we repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task. Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the video model, while further adapting them for joint modeling of images and geometry targets (image <-> geometry), enabling more data efficient and effective learning of geometry. Extensive experiments validate our method for zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming both previous task-specific and unified generative competitors while using substantially less training data. Notably, our method rivals discriminative state-of-the-art approaches trained on over 100x more data and even standouts on several benchmarks.

Related papers

• VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment [15.619170225414571]
  Video diffusion models lack explicit geometric supervision during training, leading to inconsistency artifacts.<n>We propose a geometry-based reward model that leverages pretrained geometric foundation models to evaluate multi-view consistency.<n>Our approach conducts error computation in a pointwise fashion, yielding a more physically grounded and robust error metric.
  arXiv  Detail & Related papers  (2026-03-17T09:04:10Z)
• Epipolar Geometry Improves Video Generation Models [73.44978239787501]
  3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks.<n>We explore how epipolar geometry constraints improve modern video diffusion models.<n>By bridging data-driven deep learning with classical geometric computer vision, we present a practical method for generating spatially consistent videos.
  arXiv  Detail & Related papers  (2025-10-24T16:21:37Z)
• UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation [63.90470530428842]
  In this work, we demonstrate that, through appropriate design and fine-tuning, the intrinsic consistency of video generation models can be effectively harnessed for consistent geometric estimation.<n>Our results achieve superior performance in predicting global geometric attributes in videos and can be directly applied to reconstruction tasks.
  arXiv  Detail & Related papers  (2025-05-30T12:31:59Z)
• Generative Human Geometry Distribution [49.58025398670139]
  We build upon Geometry distributions, a recently proposed representation that can model a single human geometry with high fidelity.<n>We propose a new geometry distribution model by two key techniques: encoding distributions as 2D feature maps rather than network parameters, and using SMPL models as the domain instead of Gaussian.<n> Experimental results demonstrate that our method outperforms existing state-of-the-art methods, achieving a 57% improvement in geometry quality.
  arXiv  Detail & Related papers  (2025-03-03T11:55:19Z)
• Geometry Distributions [51.4061133324376]
  We propose a novel geometric data representation that models geometry as distributions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity.
  arXiv  Detail & Related papers  (2024-11-25T04:06:48Z)
• GeoBench: Benchmarking and Analyzing Monocular Geometry Estimation Models [41.76935689355034]
  Discriminative and generative pretraining have yielded geometry estimation models with strong generalization capabilities. We build fair and strong baselines for evaluating and analyzing the geometry estimation models. We evaluate monocular geometry estimators on more challenging benchmarks for geometry estimation task with diverse scenes and high-quality annotations.
  arXiv  Detail & Related papers  (2024-06-18T14:44:12Z)
• GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image [94.56927147492738]
  We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes from single images. We show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. We propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions.
  arXiv  Detail & Related papers  (2024-03-18T17:50:41Z)
• Few-shot Image Generation via Information Transfer from the Built Geodesic Surface [2.617962830559083]
  We propose a method called Information Transfer from the Built Geodesic Surface (ITBGS) With the FAGS module, a pseudo-source domain is created by projecting image features from the training dataset into the Pre-Shape Space. We demonstrate that the proposed method consistently achieves optimal or comparable results across a diverse range of semantically distinct datasets.
  arXiv  Detail & Related papers  (2024-01-03T13:57:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.