Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalization

• URL: http://arxiv.org/abs/2608.22740v1
• Date: Mon, 24 Aug 2026 02:58:24 GMT
• Title: Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalization
• Abstract summary: Generalizable 3D Gaussian Splatting (G-3DGS) has emerged as a promising approach for novel view synthesis undersparse-view settings.<n>Toaddress these challenges, we propose SeeU, a novel G-3DGS framework.<n>SeeU achieves an average improvement of 2.44 dB in PSNR compared to recent SOTA G-3DGS methods.
• Score: 11.792866109225924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalizable 3D Gaussian Splatting (G-3DGS) has emerged as a promising approach for novel view synthesis undersparse-view settings. However, existing frameworks remain restricted by pixel-aligned Gaussian estimation, whichstruggles in partially observed or occluded regions and often leads to incomplete surfaces or structural collapse. Toaddress these challenges, we propose SeeU (Seeing the Unseen), a novel G-3DGS framework. We frame its core design asSemantic-in-Gaussian: semantic-conditioned refinement in Gaussian space. Specifically, we introduce a Cross-viewEntropy-Aware (CEA) module that aggregates multi-view semantic and geometric cues into compact embeddings. Theseembeddings guide the Conditional Gaussian Transformer, which applies residual updates to coarse Gaussians, helpingrecover under-constrained regions of partially observed structures while preserving surface consistency. Comprehensiveexperiments on multiple benchmarks demonstrate that SeeU consistently improves rendering quality and structuralcompleteness while retaining efficient feed-forward inference. Especially under challenging extrapolation settings,SeeU achieves an average improvement of 2.44 dB in PSNR compared to recent SOTA G-3DGS methods.

Related papers

• NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation [65.34304674634713]
  We introduce NG-GS, a framework for high-quality object segmentation in 3DGS that explicitly addresses boundary discretization.<n>We show that our method achieves photorealistic state-of-the-art performance, with significant gains in boundary mIoU.
  arXiv  Detail & Related papers  (2026-04-16T07:14:07Z)
• GS^2: Graph-based Spatial Distribution Optimization for Compact 3D Gaussian Splatting [50.2045409672451]
  3D Gaussian Splatting (3DGS) has demonstrated breakthrough performance in novel view synthesis and real-time rendering.<n>It is constrained by the high memory cost due to a huge number of Gaussian points.<n>We propose graph-based spatial distribution optimization for compact 3DGS.
  arXiv  Detail & Related papers  (2026-04-02T10:41:51Z)
• HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views [33.41171755717418]
  We propose HeroGS, a unified framework that establishes hierarchical guidance across the image, feature, and parameter levels.<n>We show that HeroGS achieves high-fidelity reconstructions and consistently surpasses state-of-the-art baselines under sparse-view conditions.
  arXiv  Detail & Related papers  (2026-03-01T13:35:49Z)
• COSMOS: Coherent Supergaussian Modeling with Spatial Priors for Sparse-View 3D Splatting [2.549594916296063]
  3D Gaussian Splatting (3DGS) has recently emerged as a promising approach for 3D reconstruction.<n>We propose Coherent supergaussian Modeling with Spatial Priors (COSMOS) to address this issue.<n>Our experiments on Blender and DTU show that COSMOS surpasses state-of-the-art methods in sparse-view settings without any external depth supervision.
  arXiv  Detail & Related papers  (2025-12-17T04:55:15Z)
• C3G: Learning Compact 3D Representations with 2K Gaussians [55.04010158339562]
  Recent approaches use per-pixel 3D Gaussian Splatting for reconstruction, followed by a 2D-to-3D feature lifting stage for scene understanding.<n>We propose C3G, a novel feed-forward framework that estimates compact 3D Gaussians only at essential spatial locations.
  arXiv  Detail & Related papers  (2025-12-03T17:59:05Z)
• D$^2$GS: Depth-and-Density Guided Gaussian Splatting for Stable and Accurate Sparse-View Reconstruction [73.61056394880733]
  3D Gaussian Splatting (3DGS) enables real-time, high-fidelity novel view synthesis (NVS) with explicit 3D representations.<n>We identify two key failure modes under sparse-view conditions: overfitting in regions with excessive Gaussian density near the camera, and underfitting in distant areas with insufficient Gaussian coverage.<n>We propose a unified framework D$2$GS, comprising two key components: a Depth-and-Density Guided Dropout strategy, and a Distance-Aware Fidelity Enhancement module.
  arXiv  Detail & Related papers  (2025-10-09T17:59:49Z)
• Accurate and Complete Surface Reconstruction from 3D Gaussians via Direct SDF Learning [5.604709769018076]
  3D Gaussian Splatting (3DGS) has emerged as a powerful paradigm for photorealistic view synthesis.<n>We propose DiGS, a unified framework that embeds Signed Distance Field (SDF) learning directly into the 3DGS pipeline.<n>We show that DiGS consistently improves reconstruction accuracy and completeness while retaining high fidelity.
  arXiv  Detail & Related papers  (2025-09-09T08:17:46Z)
• TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views [18.050257821756148]
  TSGaussian is a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in novel view synthesis tasks.<n>Our approach prioritizes computational resources on designated targets while minimizing background allocation.<n>Extensive experiments demonstrate that TSGaussian outperforms state-of-the-art methods on three standard datasets.
  arXiv  Detail & Related papers  (2024-12-13T11:26:38Z)
• MonoGSDF: Exploring Monocular Geometric Cues for Gaussian Splatting-Guided Implicit Surface Reconstruction [86.87464903285208]
  We introduce MonoGSDF, a novel method that couples primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction.<n>To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization.<n>Experiments on real-world datasets outperforms prior methods while maintaining efficiency.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• PEP-GS: Perceptually-Enhanced Precise Structured 3D Gaussians for View-Adaptive Rendering [3.1006820631993515]
  3D Gaussian Splatting (3D-GS) has achieved significant success in real-time, high-quality 3D scene rendering.<n>We introduce PEP-GS, a perceptually-enhanced framework that dynamically predicts Gaussian attributes, including opacity, color, and covariance.<n>We show that PEP-GS outperforms state-of-the-art methods, particularly in challenging scenarios involving view-dependent effects and fine-scale details.
  arXiv  Detail & Related papers  (2024-11-08T17:42:02Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.<n>Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields [100.90743697473232]
  Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering.<n>Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images.<n>We propose a view synthesis framework based on 3D Gaussian Splatting, enabling scene reconstruction from sparse views.
  arXiv  Detail & Related papers  (2024-10-15T08:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.