What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility

• URL: http://arxiv.org/abs/2607.09503v1
• Date: Fri, 10 Jul 2026 15:17:02 GMT
• Title: What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility
• Abstract summary: VGGT implicitly encodes co-visibility as an emergent behavior.<n>We introduce specialization Co-VGGT, which freezes VGGT and trains only a lightweight layer-wise mixture-of-experts head.<n>On the Co-VisiON benchmark, Co-VGGT surpasses the human annotation baseline and improves over prior work by more than 25% pairwise and 10% multiview.
• Score: 14.143547667315419
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A fundamental challenge in 3D reconstruction and robotic localization is co-visibility: determining which image pairs share overlapping visible surfaces, particularly in scenarios with minimal overlap. We demonstrate that VGGT implicitly encodes co-visibility as an emergent behavior: without any supervision for this task, its internal representations exhibit a clear hierarchical structure mirroring that of large language models, i.e. early layers build a 3D-aware scene representation, while late layers act as dedicated co-visibility reasoners. In particular, we identify layer L17 as a negative anchor that consistently routes non-co-visible pairs for this backbone, regardless of the evaluation setting, providing task-grounded evidence of layer specialization in a geometry-grounded foundation model. Building on this, we introduce Co-VGGT, which freezes VGGT and trains only a lightweight layer-wise mixture-of-experts head (less than 7.5M parameters) to classify co-visibility from RGB alone, treating each layer as a specialized expert whose geometric abstraction is adaptively weighted per input pair. On the Co-VisiON benchmark, Co-VGGT surpasses the human annotation baseline and improves over prior work by more than 25% pairwise and 10% multiview. Pairwise predictions are well-calibrated (ECE=0.030), enabling direct use as edge weights in visibility graphs for downstream SfM and SLAM pipelines without post-hoc correction. Code and data are available.

Related papers

• Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models [12.061080782435013]
  Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization.<n> enforcing explicit multi-view geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining.<n>We propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth.
  arXiv  Detail & Related papers  (2026-08-11T09:28:50Z)
• GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation [75.85672467847631]
  We introduce the Geometry-Aware BEV (GA-BEV) - a compact, 3D-grounded feature representation.<n>We construct BEV spatial maps from RGB-D inputs by projecting visual features into 3D space.<n>We incorporate features from a pretrained 3D foundation model into the BEV space, injecting structural priors learned from large-scale 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-05-21T06:20:17Z)
• VGGT-Segmentor: Geometry-Enhanced Cross-View Segmentation [17.578132395389755]
  We introduce VGGT-Segmentor (VGGT-S), a geometric framework that unifies pixel-accurate semantic segmentation.<n>On the Ego-Exo4D benchmark, VGGT-S sets a new state-of-the-art, achieving 67.7% and 68.0% average IoU for Ego to Exo and Exo to Ego tasks, respectively.
  arXiv  Detail & Related papers  (2026-04-15T08:05:13Z)
• LG-HCC: Local Geometry-Aware Hierarchical Context Compression for 3D Gaussian Splatting [77.81227097905865]
  Anchor-based 3DGS compression schemes reduce gaussian redundancy through some advanced context models.<n>We propose a Local Geometry-aware Hierarchical Context Compression framework for 3DGS that incorporates inter-anchor geometric correlations into anchor pruning and entropy coding for compact representation.<n>Experiments show that LG-HCC effectively alleviates structural preservation issues,achieving superior geometric integrity and rendering fidelity while reducing storage by up to 30.85x compared to the Scaffold-GS baseline on the Mip-NeRF360 dataset.
  arXiv  Detail & Related papers  (2026-03-30T13:39:35Z)
• GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence [16.503835991826367]
  We present GLASS, a framework that bridges the gap by integrating geometric spectral analysis with rich semantic priors from vision-language foundation models.<n> GLASS introduces three key innovations: (i) a view-consistent strategy that enables robust multi-view visual feature extraction from powerful vision foundation models.<n>It achieves average geodesic errors of 0.21, 4.5, and 5.6 on the inter-class benchmark SMAL and non-isometric benchmarks SMAL and TOPKIDS, reducing errors from URSSM baselines of 0.49, 6.0, and 8.9 by 57%, 25%, and 37%, respectively.
  arXiv  Detail & Related papers  (2026-03-08T14:33:50Z)
• MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation [59.75554954111619]
  We introduce Multi-view 3D Referring Expression (MV-3DRES), where the model must recover scene structure and segment the referred object directly from sparse multi-view images.<n>We propose the Multimodal Visual Geometry Grounded Transformer (MVGGT), an efficient end-to-end framework that integrates language information into sparse-view geometric reasoning.<n>Experiments show that MVGGT establishes the first strong baseline and achieves both high accuracy and fast inference, outperforming existing alternatives.
  arXiv  Detail & Related papers  (2026-01-11T11:44:07Z)
• On Geometric Understanding and Learned Data Priors in VGGT [38.8968170074396]
  The Visual Geometry Grounded Transformer (VGGT) is a 3D foundation model that infers camera geometry and scene structure in a single feed-forward pass.<n>We conduct a systematic analysis of VGGT's internal mechanisms to uncover whether geometric understanding emerges within its representations.
  arXiv  Detail & Related papers  (2025-12-12T12:11:57Z)
• GLaD: Geometric Latent Distillation for Vision-Language-Action Models [106.53332923530245]
  We introduce GLaD, a geometry-aware Vision-Language-Action (VLA) framework that incorporates 3D geometric priors during pretraining through knowledge distillation.<n>GLaD achieves 94.1% average success rate across four LIBERO task suites, outperforming UniVLA (92.5%) which uses identical pretraining data.
  arXiv  Detail & Related papers  (2025-12-10T13:07:27Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• FastVGGT: Training-Free Acceleration of Visual Geometry Transformer [83.67766078575782]
  VGGT is a state-of-the-art feed-forward visual geometry model.<n>We propose FastVGGT, which leverages token merging in the 3D domain through a training-free mechanism for accelerating VGGT.<n>With 1000 input images, FastVGGT achieves a 4x speedup over VGGT while mitigating error accumulation in long-sequence scenarios.
  arXiv  Detail & Related papers  (2025-09-02T17:54:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.