IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation

• URL: http://arxiv.org/abs/2605.16258v2
• Date: Thu, 21 May 2026 03:42:22 GMT
• Title: IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation
• Abstract summary: We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images.<n> IVGT learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions.<n>It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints.
• Score: 76.36174247570716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing coherent 3D geometry and appearance from unposed multi-view images is a fundamental yet challenging problem in computer vision. Most existing visual geometry foundation models predict explicit geometry by regressing pixel-aligned pointmaps, often suffering from redundancy and limited geometric continuity. We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images. This formulation learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions, retrieving local features to predict signed distance (SDF) values and colors using lightweight decoders. It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints. We train IVGT via multi-dataset joint optimization with 2D supervision and 3D geometric regularization. IVGT demonstrates generalization across scenes and achieves strong performance on various tasks, including mesh and point cloud reconstruction, novel view synthesis, depth and surface normal estimation, and camera pose estimation.

Related papers

• ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views [9.257489669927624]
  ReconSplat is a feed-forward model for 3D scene reconstruction.<n>It provides both geometrically aligned novel views and sharp depth estimates.<n>It produces both photorealistic novel views and accurate depth maps on real-world benchmarks.
  arXiv  Detail & Related papers  (2026-08-28T21:53:58Z)
• Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction [56.51413081953813]
  SurfSVR is a novel sparse voxel reconstruction paradigm that treats 2D surface priors as explicit 3D geometric regularizers.<n>It consistently improves sparse voxel reconstruction across scenes with substantially different visibility and geometry characteristics.
  arXiv  Detail & Related papers  (2026-08-12T11:25:31Z)
• IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer [64.5765465030666]
  We propose IGGT4D, a streaming instance-grounded geometry Transformer for online 4D scene understanding.<n> IGGT4D processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally updates a unified representation of camera motion, geometry, and object identity.<n> Experiments on 3D reconstruction, pose estimation, instance spatial tracking, and open-vocabulary segmentation demonstrate that IGGT4D outperforms existing streaming baselines.
  arXiv  Detail & Related papers  (2026-07-21T16:00:01Z)
• Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video [69.94904173314505]
  We propose Ground4D, a geometry-grounded framework built on two stages.<n>First, we reconstruct multi-view-consistent 3D geometry and camera poses from monocular video.<n>Second, we conduct geometry-consistency-aware refinement via dynamic Gaussian Splatting.
  arXiv  Detail & Related papers  (2026-06-27T09:27:42Z)
• GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion [36.691978700341515]
  GeoFace is a geometry-constrained multi-view diffusion framework for consistent face generation from a single input.<n>Experiments on RenderMe-360 and NeRSemble demonstrate that GeoFace consistently outperforms existing methods in both visual quality and cross-view geometric consistency.
  arXiv  Detail & Related papers  (2026-06-26T02:31:06Z)
• World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible [34.47550981952829]
  We introduce World Tracing, a generative pixel-aligned geometry representation that predicts 3D points aligned with observed pixels.<n>For each input pixel, World Tracing predicts an ordered stack of camera-space 3D points, where the first layer represents the visible surface.<n>We instantiate this representation with a world-tracing diffusion transformer, WT-DiT, which treats multiple geometry layers as separate denoising tokens.
  arXiv  Detail & Related papers  (2026-06-11T17:52:48Z)
• Face Anything: 4D Face Reconstruction from Any Image Sequence [49.395407357499074]
  We present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction.<n>By jointly predicting depth and canonical coordinates, our method enables accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking.
  arXiv  Detail & Related papers  (2026-04-21T17:22:39Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• GGPT: Geometry Grounded Point Transformer [22.64445696362087]
  We introduce the Geometry-Grounded Point Transformer (GGPT), a framework that augments feed-forward reconstruction with reliable sparse geometric guidance.<n>Building on this foundation, we propose a geometry-guided 3D point transformer that refines dense point maps under explicit partial-geometry supervision.
  arXiv  Detail & Related papers  (2026-03-11T18:00:04Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• WorldMirror: Universal 3D World Reconstruction with Any-Prior Prompting [51.69408870574092]
  We present WorldMirror, an all-in-one, feed-forward model for versatile 3D geometric prediction tasks.<n>Our framework flexibly integrates diverse geometric priors, including camera poses, intrinsics, and depth maps.<n>WorldMirror achieves state-of-the-art performance across diverse benchmarks from camera, point map, depth, and surface normal estimation to novel view synthesis.
  arXiv  Detail & Related papers  (2025-10-12T17:59:09Z)
• Dens3R: A Foundation Model for 3D Geometry Prediction [44.13431776180547]
  Dens3R is a 3D foundation model designed for joint geometric dense prediction.<n>By integrating image-pair matching features with intrinsic invariance modeling, Dens3R accurately regresses multiple geometric quantities.
  arXiv  Detail & Related papers  (2025-07-22T07:22:30Z)
• G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images [45.66479596827045]
  We propose a Geometry-enhanced NeRF (G-NeRF), which seeks to enhance the geometry priors by a geometry-guided multi-view synthesis approach. To tackle the absence of multi-view supervision for single-view images, we design the depth-aware training approach.
  arXiv  Detail & Related papers  (2024-04-11T04:58:18Z)
• SuperPrimitive: Scene Reconstruction at a Primitive Level [23.934492494774116]
  Joint camera pose and dense geometry estimation from a set of images or a monocular video remains a challenging problem. Most dense incremental reconstruction systems operate directly on image pixels and solve for their 3D positions using multi-view geometry cues. We address this issue with a new image representation which we call a SuperPrimitive.
  arXiv  Detail & Related papers  (2023-12-10T13:44:03Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.