GGPT: Geometry Grounded Point Transformer

• URL: http://arxiv.org/abs/2603.11174v1
• Date: Wed, 11 Mar 2026 18:00:04 GMT
• Title: GGPT: Geometry Grounded Point Transformer
• Abstract summary: We introduce the Geometry-Grounded Point Transformer (GGPT), a framework that augments feed-forward reconstruction with reliable sparse geometric guidance.<n>Building on this foundation, we propose a geometry-guided 3D point transformer that refines dense point maps under explicit partial-geometry supervision.
• Score: 22.64445696362087
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent feed-forward networks have achieved remarkable progress in sparse-view 3D reconstruction by predicting dense point maps directly from RGB images. However, they often suffer from geometric inconsistencies and limited fine-grained accuracy due to the absence of explicit multi-view constraints. We introduce the Geometry-Grounded Point Transformer (GGPT), a framework that augments feed-forward reconstruction with reliable sparse geometric guidance. We first propose an improved Structure-from-Motion pipeline based on dense feature matching and lightweight geometric optimisation to efficiently estimate accurate camera poses and partial 3D point clouds from sparse input views. Building on this foundation, we propose a geometry-guided 3D point transformer that refines dense point maps under explicit partial-geometry supervision using an optimised guidance encoding. Extensive experiments demonstrate that our method provides a principled mechanism for integrating geometric priors with dense feed-forward predictions, producing reconstructions that are both geometrically consistent and spatially complete, recovering fine structures and filling gaps in textureless areas. Trained solely on ScanNet++ with VGGT predictions, GGPT generalises across architectures and datasets, substantially outperforming state-of-the-art feed-forward 3D reconstruction models in both in-domain and out-of-domain settings.

Related papers

• SphereSOD: Geometry-Structure Coupled Learning for 360 Salient Object Detection [75.77856258186854]
  360 salient object detection aims to accurately segment salient regions across a full field of view.<n>Existing methods mainly focus on compensating projection distortion while overlooking the interaction between panoramic geometry and salient object structure.<n>We propose SphereSOD, an ERP-native framework that couples panoramic geometry with evolving salient structures.
  arXiv  Detail & Related papers  (2026-09-07T14:47:54Z)
• Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction [56.51413081953813]
  SurfSVR is a novel sparse voxel reconstruction paradigm that treats 2D surface priors as explicit 3D geometric regularizers.<n>It consistently improves sparse voxel reconstruction across scenes with substantially different visibility and geometry characteristics.
  arXiv  Detail & Related papers  (2026-08-12T11:25:31Z)
• Visual Geometry Foundation-Aware Gaussians for Single-Frame Surround-View Driving Reconstruction [44.54907293631917]
  Single-frame surround-view reconstruction faces severe geometric instability and rendering artifacts due to minimal inter-camera overlap.<n>We propose VGGD, a visual geometry foundation-aware 3D Gaussian Splatting framework for feed-forward surround-view driving reconstruction.
  arXiv  Detail & Related papers  (2026-08-11T09:04:33Z)
• Confidence matters: Leveraging Multi-view Geometric Priors for GS-based Reconstruction [28.891336184070223]
  3D Gaussian splatting (3DGS) has emerged as a widely-used tool for novel view synthesis, offering real-time rendering in a sparse representation.<n>We investigate the integration of geometric priors, in the form of predicted normal and depth maps, into the 3DGS framework to improve the reconstruction quality.
  arXiv  Detail & Related papers  (2026-08-06T14:50:29Z)
• TopoGS: Planar Reconstruction via Topology-aware 3D Gaussian Splatting [15.690820181002517]
  We introduce TopoGS, the first 3DGS framework to explicitly integrate both planar and topological constraints for coherent 3D reconstruction.<n>Specifically, we extract global 2D topological relationships from multi-view image segmentations and anchor Gaussian primitives to these structural elements.<n>By enforcing strict multi-view consistency alongside these topological constraints, our method significantly mitigates geometric misalignments and produces connected, structured 3D models.
  arXiv  Detail & Related papers  (2026-07-18T14:27:49Z)
• IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation [76.36174247570716]
  We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images.<n> IVGT learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions.<n>It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2026-05-15T17:59:57Z)
• TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction [57.46712611558817]
  3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass.<n>Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry.<n>We propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies.
  arXiv  Detail & Related papers  (2025-12-02T02:22:20Z)
• Rethinking Multimodal Point Cloud Completion: A Completion-by-Correction Perspective [8.276620253870338]
  Point cloud completion aims to reconstruct complete 3D shapes from partial observations.<n>Most methods still follow a Completion-by-Inpainting paradigm.<n>We propose Completion-by-Correction, which begins with a complete shape prior and performs feature-space correction to align it with the partial observation.
  arXiv  Detail & Related papers  (2025-11-15T11:51:13Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior [53.762256749551284]
  We identify accurate geometry as the fundamental prerequisite for effectively exploiting generative models to enhance 3D scene reconstruction.<n>We incorporate this geometry guidance throughout the generative pipeline to improve visibility mask estimation, guide novel view selection, and enhance multi-view consistency when inpainting with video diffusion models.<n>Our method naturally supports single-view inputs and unposed videos, with strong generalizability in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2025-10-14T03:06:28Z)
• Dens3R: A Foundation Model for 3D Geometry Prediction [44.13431776180547]
  Dens3R is a 3D foundation model designed for joint geometric dense prediction.<n>By integrating image-pair matching features with intrinsic invariance modeling, Dens3R accurately regresses multiple geometric quantities.
  arXiv  Detail & Related papers  (2025-07-22T07:22:30Z)
• LineGS : 3D Line Segment Representation on 3D Gaussian Splatting [0.0]
  LineGS is a novel method that combines geometry-guided 3D line reconstruction with a 3D Gaussian splatting model.<n>The results show significant improvements in both geometric accuracy and model compactness compared to baseline methods.
  arXiv  Detail & Related papers  (2024-11-30T13:29:36Z)
• Geometry-guided Feature Learning and Fusion for Indoor Scene Reconstruction [14.225228781008209]
  This paper proposes a novel geometry integration mechanism for 3D scene reconstruction. Our approach incorporates 3D geometry at three levels, i.e. feature learning, feature fusion, and network supervision.
  arXiv  Detail & Related papers  (2024-08-28T08:02:47Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.