Leveling3D: Leveling Up 3D Reconstruction with Feed-Forward 3D Gaussian Splatting and Geometry-Aware Generation

• URL: http://arxiv.org/abs/2603.16211v1
• Date: Tue, 17 Mar 2026 07:40:45 GMT
• Title: Leveling3D: Leveling Up 3D Reconstruction with Feed-Forward 3D Gaussian Splatting and Geometry-Aware Generation
• Abstract summary: We introduce Leveling3D, a novel pipeline that integrates feed-forward 3D reconstruction with geometrical-consistent generation.<n>We achieve SOTA performance on public datasets, including tasks such as novel-view synthesis and depth estimation.
• Score: 15.735997729565987
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Feed-forward 3D reconstruction has revolutionized 3D vision, providing a powerful baseline for downstream tasks such as novel-view synthesis with 3D Gaussian Splatting. Previous works explore fixing the corrupted rendering results with a diffusion model. However, they lack geometric concern and fail at filling the missing area on the extrapolated view. In this work, we introduce Leveling3D, a novel pipeline that integrates feed-forward 3D reconstruction with geometrical-consistent generation to enable holistic simultaneous reconstruction and generation. We propose a geometry-aware leveling adapter, a lightweight technique that aligns internal knowledge in the diffusion model with the geometry prior from the feed-forward model. The leveling adapter enables generation on the artifact area of the extrapolated novel views caused by underconstrained regions of the 3D representation. Specifically, to learn a more diverse distributed generation, we introduce the palette filtering strategy for training, and a test-time masking refinement to prevent messy boundaries along the fixing regions. More importantly, the enhanced extrapolated novel views from Leveling3D could be used as the inputs for feed-forward 3DGS, leveling up the 3D reconstruction. We achieve SOTA performance on public datasets, including tasks such as novel-view synthesis and depth estimation.

Related papers

• Pano3D: Unified 3D Reconstruction and Panoptic Segmentation [68.18993232814061]
  We introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework.<n>We build on existing 3D reconstruction models and augment them with a set-based mask decoder.<n>Our method achieves state-of-the-art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets.
  arXiv  Detail & Related papers  (2026-06-12T09:43:13Z)
• EA3D: Online Open-World 3D Object Extraction from Streaming Videos [55.48835711373918]
  We present ExtractAnything3D (EA3D), a unified online framework for open-world 3D object extraction.<n>Given a streaming video, EA3D dynamically interprets each frame using vision-language and 2D vision foundation encoders to extract object-level knowledge.<n>A recurrent joint optimization module directs the model's attention to regions of interest, simultaneously enhancing both geometric reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2025-10-29T03:56:41Z)
• From Volume Rendering to 3D Gaussian Splatting: Theory and Applications [5.318207476146788]
  This tutorial provides a concise yet comprehensive overview of the 3DGS pipeline.<n>It starts from its splatting formulation and then explores the main efforts in addressing its limitations.<n>We survey a range of applications that leverage 3DGS for surface reconstruction, avatar modeling, animation, and content generation.
  arXiv  Detail & Related papers  (2025-10-20T20:52:41Z)
• EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis [61.1662426227688]
  Existing NeRF and 3DGS-based methods show promising results in achieving photorealistic renderings but require slow, per-scene optimization.<n>We introduce EVolSplat, an efficient 3D Gaussian Splatting model for urban scenes that works in a feed-forward manner.
  arXiv  Detail & Related papers  (2025-03-26T02:47:27Z)
• Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models [65.90387371072413]
  We introduce Difix3D+, a novel pipeline designed to enhance 3D reconstruction and novel-view synthesis.<n>At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views.
  arXiv  Detail & Related papers  (2025-03-03T17:58:33Z)
• TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models [69.0220314849478]
  TripoSG is a new streamlined shape diffusion paradigm capable of generating high-fidelity 3D meshes with precise correspondence to input images.<n>The resulting 3D shapes exhibit enhanced detail due to high-resolution capabilities and demonstrate exceptional fidelity to input images.<n>To foster progress and innovation in the field of 3D generation, we will make our model publicly available.
  arXiv  Detail & Related papers  (2025-02-10T16:07:54Z)
• LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation [73.36690511083894]
  This paper introduces a novel framework called LN3Diff to address a unified 3D diffusion pipeline. Our approach harnesses a 3D-aware architecture and variational autoencoder to encode the input image into a structured, compact, and 3D latent space. It achieves state-of-the-art performance on ShapeNet for 3D generation and demonstrates superior performance in monocular 3D reconstruction and conditional 3D generation.
  arXiv  Detail & Related papers  (2024-03-18T17:54:34Z)
• Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network [42.16520614686877]
  3D-GANs exhibit artifacts in their 3D geometrical modeling, such as mesh imperfections and holes. These shortcomings are primarily attributed to the limited availability of annotated 3D data. We present a Self-Supervised Learning technique tailored as an auxiliary loss for any 3D-GAN.
  arXiv  Detail & Related papers  (2023-12-19T04:55:33Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)
• Learning Neural Radiance Fields from Multi-View Geometry [1.1011268090482573]
  We present a framework, called MVG-NeRF, that combines Multi-View Geometry algorithms and Neural Radiance Fields (NeRF) for image-based 3D reconstruction. NeRF has revolutionized the field of implicit 3D representations, mainly due to a differentiable rendering formulation that enables high-quality and geometry-aware novel view synthesis.
  arXiv  Detail & Related papers  (2022-10-24T08:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.