Fus3D: Decoding Consolidated 3D Geometry from Feed-forward Geometry Transformer Latents

• URL: http://arxiv.org/abs/2603.25827v1
• Date: Thu, 26 Mar 2026 18:45:04 GMT
• Title: Fus3D: Decoding Consolidated 3D Geometry from Feed-forward Geometry Transformer Latents
• Abstract summary: We propose a feed-forward method for dense Signed Distance Field (SDF) regression from unstructured image collections in less than three seconds.<n>Our key insight is that the intermediate feature space of pretrained multi-view feed-forward geometry transformers already encodes a powerful joint world representation.<n>We instead perform 3D extraction directly from geometry transformer features via learned volumetric extraction.
• Score: 10.400785196849647
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose a feed-forward method for dense Signed Distance Field (SDF) regression from unstructured image collections in less than three seconds, without camera calibration or post-hoc fusion. Our key insight is that the intermediate feature space of pretrained multi-view feed-forward geometry transformers already encodes a powerful joint world representation; yet, existing pipelines discard it, routing features through per-view prediction heads before assembling 3D geometry post-hoc, which discards valuable completeness information and accumulates inaccuracies. We instead perform 3D extraction directly from geometry transformer features via learned volumetric extraction: voxelized canonical embeddings that progressively absorb multi-view geometry information through interleaved cross- and self-attention into a structured volumetric latent grid. A simple convolutional decoder then maps this grid to a dense SDF. We additionally propose a scalable, validity-aware supervision scheme directly using SDFs derived from depth maps or 3D assets, tackling practical issues like non-watertight meshes. Our approach yields complete and well-defined distance values across sparse- and dense-view settings and demonstrates geometrically plausible completions. Code and further material can be found at https://lorafib.github.io/fus3d.

Related papers

• Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models [27.12023248183448]
  Generative models for 3D scenes must rely on strong geometric priors to produce coherent outputs from sparse inputs.<n>We bridge these two paradigms by performing flow matching directly in VGGT's latent space.<n>On RealEstate10K, ScanNet++ and ETH3D, our method achieves strong performance against recent scene generation baselines.
  arXiv  Detail & Related papers  (2026-07-21T14:06:04Z)
• TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction [65.03268438396556]
  TriSplat is a feed-forward reconstruction network that represents scenes with oriented triangle primitives.<n>It exports simulation-ready mesh scenes from a single forward pass.<n>Experiments on RealEstate10K and DL3DV show that this representation produces more geometry-faithful reconstructions than Gaussian feed-forward baselines.
  arXiv  Detail & Related papers  (2026-05-25T17:59:53Z)
• GGPT: Geometry Grounded Point Transformer [22.64445696362087]
  We introduce the Geometry-Grounded Point Transformer (GGPT), a framework that augments feed-forward reconstruction with reliable sparse geometric guidance.<n>Building on this foundation, we propose a geometry-guided 3D point transformer that refines dense point maps under explicit partial-geometry supervision.
  arXiv  Detail & Related papers  (2026-03-11T18:00:04Z)
• A 3D mesh convolution-based autoencoder for geometry compression [0.769971486557519]
  We introduce a novel 3D mesh convolution-based autoencoder for geometry compression, able to deal with irregular mesh data without requiring neither preprocessing nor manifold/watertightness conditions.<n>The proposed approach extracts meaningful latent representations by learning features directly from the mesh faces, while preserving connectivity through dedicated pooling and unpooling operations.
  arXiv  Detail & Related papers  (2026-03-02T17:42:58Z)
• PatchAlign3D: Local Feature Alignment for Dense 3D Shape understanding [67.15800065888887]
  Current foundation models for 3D shapes excel at global tasks (retrieval, classification) but transfer poorly to local part-level reasoning.<n>We introduce an encoder-only 3D model that produces language-aligned patch-level features directly from point clouds.<n>Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering.
  arXiv  Detail & Related papers  (2026-01-05T18:55:45Z)
• TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction [57.46712611558817]
  3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass.<n>Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry.<n>We propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies.
  arXiv  Detail & Related papers  (2025-12-02T02:22:20Z)
• PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting [56.188624157291024]
  We introduce PLANA3R, a pose-free framework for metric Planar 3D Reconstruction from unposed two-view images.<n>Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision.<n>We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments.
  arXiv  Detail & Related papers  (2025-10-21T15:15:33Z)
• Dens3R: A Foundation Model for 3D Geometry Prediction [44.13431776180547]
  Dens3R is a 3D foundation model designed for joint geometric dense prediction.<n>By integrating image-pair matching features with intrinsic invariance modeling, Dens3R accurately regresses multiple geometric quantities.
  arXiv  Detail & Related papers  (2025-07-22T07:22:30Z)
• POC-SLT: Partial Object Completion with SDF Latent Transformers [1.5999407512883512]
  3D geometric shape completion hinges on representation learning and a deep understanding of geometric data. We propose a transformer operating on the latent space representing Signed Distance Fields (SDFs) Instead of a monolithic volume, the SDF of an object is partitioned into smaller high-resolution patches leading to a sequence of latent codes.
  arXiv  Detail & Related papers  (2024-11-08T09:13:20Z)
• Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction [2.741266294612776]
  We present a novel method for reconstructing 3D surfaces using a multi-beam imaging sonar. We integrate the intensities measured by the sonar from different viewpoints for fixed cell positions in a 3D grid. We train convolutional neural networks that allow us to predict the signed distance and direction to the nearest surface for each cell.
  arXiv  Detail & Related papers  (2022-06-06T18:24:14Z)
• Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
  The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies. We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
  arXiv  Detail & Related papers  (2021-12-14T13:14:24Z)
• KAPLAN: A 3D Point Descriptor for Shape Completion [80.15764700137383]
  KAPLAN is a 3D point descriptor that aggregates local shape information via a series of 2D convolutions. In each of those planes, point properties like normals or point-to-plane distances are aggregated into a 2D grid and abstracted into a feature representation with an efficient 2D convolutional encoder. Experiments on public datasets show that KAPLAN achieves state-of-the-art performance for 3D shape completion.
  arXiv  Detail & Related papers  (2020-07-31T21:56:08Z)
• Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion [53.885984328273686]
  Implicit Feature Networks (IF-Nets) deliver continuous outputs, can handle multiple topologies, and complete shapes for missing or sparse input data. IF-Nets clearly outperform prior work in 3D object reconstruction in ShapeNet, and obtain significantly more accurate 3D human reconstructions.
  arXiv  Detail & Related papers  (2020-03-03T11:14:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.