Visual Geometry Transformer in the Wild: Distractor-Free 3D Reconstruction

• URL: http://arxiv.org/abs/2606.22787v1
• Date: Mon, 22 Jun 2026 02:52:22 GMT
• Title: Visual Geometry Transformer in the Wild: Distractor-Free 3D Reconstruction
• Abstract summary: Current end-to-end multi-view 3D reconstruction methods achieve impressive results, but rely on a restrictive static assumption.<n>This reliance on idealized inputs causes even the most advanced methods to fail in real-world settings.<n>We propose Visual Geometry Transformer in the Wild, an end-to-end framework for robust reconstruction from inconsistent views.
• Score: 77.75911117889915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current end-to-end multi-view 3D reconstruction methods achieve impressive results, but rely on a restrictive static assumption: the scenes is entire distractor-free with perfect cross-view geometry. This reliance on idealized inputs causes even the most advanced methods to fail in real-world settings, where transient distractors and occlusions present. To address this, we propose Visual Geometry Transformer in the Wild (VGTW), an end-to-end framework for robust reconstruction from inconsistent views. At its core, we isolate and suppress distractor-affected regions while preserving the consistent components across views. Specifically, we introduce a Distractor-aware Training (DAT) strategy that separates clean features from distractor-contaminated ones in the attention mechanism while enforcing feature consistency across images. To enable this, we train the model with an auxiliary mask prediction head, using supervision from a new dataset we collected with pixel-level distractor masks. The resulting VGTW model is a feed-forward network that directly outputs clean, distractor-free point clouds. Remarkably, it requires no additional 3D supervision, remains computationally efficient, and is compatible with existing pipelines. Extensive experiments validate our approach, demonstrating state-of-the-art performance and robust generalization in diverse, real-world scenarios.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Filling the Unseen: Scene Extrapolation via 3D Gaussian Splatting [10.582831709592789]
  3D Gaussian Splatting reconstruction within training view distribution degrades on out-of-distribution novel views, exhibiting holes in unobserved regions and artifacts in observable areas.<n>Recent works formulate this task as extrapolation and conditioning and try to address it with generative models, but remain limited in extrapolation scale and quality.<n>We propose a holistic framework for extrapolation and conditioning, with an independent camera view detection mechanism to enable parallel conflict-free extrapolation.
  arXiv  Detail & Related papers  (2026-09-06T19:02:27Z)
• Sparse auto-regressive modeling for scene generation from multi-view images [20.272184280044083]
  We introduce SPAR3S, a sparse voxel-aligned 3D latent generative model for conditional scene completion.<n>Our key insight is to formulate 3D scene generation in a structured, compact, voxel-aligned 3D latent space.<n>We demonstrate the effectiveness of our method on synthetic indoor scenes, achieving higher novel-view quality than prior work.
  arXiv  Detail & Related papers  (2026-09-03T14:41:34Z)
• View-Adaptive Renderer for View-Consistent 2D-to-3D Generation [2.8951915538170288]
  Reconstructing 3D shapes from a single image remains a fundamental yet challenging problem in computer vision.<n>Traditional monocular 3D generation pipelines typically synthesize multiple views from a single input image before applying Neural Radiance Field (NeRF)-based reconstruction.<n>We propose a novel viewpoint-adaptive rendering neural framework that enables robust 3D reconstruction even when given partially inconsistent multi-view inputs.
  arXiv  Detail & Related papers  (2026-08-10T04:28:08Z)
• TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction [65.03268438396556]
  TriSplat is a feed-forward reconstruction network that represents scenes with oriented triangle primitives.<n>It exports simulation-ready mesh scenes from a single forward pass.<n>Experiments on RealEstate10K and DL3DV show that this representation produces more geometry-faithful reconstructions than Gaussian feed-forward baselines.
  arXiv  Detail & Related papers  (2026-05-25T17:59:53Z)
• Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation [63.01022057888141]
  Asset Harvester is an image-to-3D model and end-to-end pipeline that converts sparse, in-thewild object observations from real driving logs into complete, simulation-ready assets.<n>SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges.
  arXiv  Detail & Related papers  (2026-04-20T16:20:57Z)
• To View Transform or Not to View Transform: NeRF-based Pre-training Perspective [15.748407383575604]
  We propose a novel NeRF-Resembled Point-based 3D detector that can learn continuous 3D representation and thus avoid the misaligned priors from view transformation.<n>NeRP3D preserves the pre-trained NeRF network regardless of the tasks, inheriting the principle of continuous 3D representation learning.
  arXiv  Detail & Related papers  (2026-03-30T06:46:34Z)
• Emergent Outlier View Rejection in Visual Geometry Grounded Transformers [45.83800698097105]
  "Noisy" images-irrelevant inputs with little or no view overlap with others hinder Reliable 3D reconstruction from in-the-wild image collections.<n>Traditional Structure-from-Motion pipelines handle such cases through geometric verification and outlier rejection.<n>In this paper, we discover that the existing feed-forward reconstruction model, VGGT, can inherently distinguish distractor images.
  arXiv  Detail & Related papers  (2025-12-03T17:48:25Z)
• Understanding Multi-View Transformers [18.573401296925844]
  Multi-view transformers such as DUSt3R are revolutionizing 3D vision by solving 3D tasks in a feed-forward manner.<n>Here, we present an approach for probing and visualizing 3D representations from the residual connections of the multi-view transformers' layers.
  arXiv  Detail & Related papers  (2025-10-28T19:19:35Z)
• IGFuse: Interactive 3D Gaussian Scene Reconstruction via Multi-Scans Fusion [15.837932667195037]
  IGFuse is a novel framework that reconstructs interactive Gaussian scene by fusing observations from multiple scans.<n>Our method constructs segmentation aware Gaussian fields and enforces bi-directional photometric and semantic consistency across scans.<n>IGFuse enables high fidelity rendering and object level scene manipulation without dense observations or complex pipelines.
  arXiv  Detail & Related papers  (2025-08-18T17:59:47Z)
• Robust 3D Gaussian Splatting for Novel View Synthesis in Presence of Distractors [44.55317154371679]
  3D Gaussian Splatting has shown impressive novel view synthesis results. It is vulnerable to dynamic objects polluting the input data of an otherwise static scene, so called distractors. We show that our approach is robust to various distractors and strongly improves rendering quality on distractor-polluted scenes.
  arXiv  Detail & Related papers  (2024-08-21T15:21:27Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.