RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images

• URL: http://arxiv.org/abs/2503.14198v1
• Date: Tue, 18 Mar 2025 12:18:34 GMT
• Title: RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images
• Authors: Junjin Xiao, Qing Zhang, Yonewei Nie, Lei Zhu, Wei-Shi Zheng,
• Abstract summary: RoGSplat is a novel approach for synthesizing high-fidelity novel views of unseen human from sparse multi-view images.<n>Our method outperforms state-of-the-art methods in novel view synthesis and cross-dataset generalization.
• Score: 39.03889696169877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents RoGSplat, a novel approach for synthesizing high-fidelity novel views of unseen human from sparse multi-view images, while requiring no cumbersome per-subject optimization. Unlike previous methods that typically struggle with sparse views with few overlappings and are less effective in reconstructing complex human geometry, the proposed method enables robust reconstruction in such challenging conditions. Our key idea is to lift SMPL vertices to dense and reliable 3D prior points representing accurate human body geometry, and then regress human Gaussian parameters based on the points. To account for possible misalignment between SMPL model and images, we propose to predict image-aligned 3D prior points by leveraging both pixel-level features and voxel-level features, from which we regress the coarse Gaussians. To enhance the ability to capture high-frequency details, we further render depth maps from the coarse 3D Gaussians to help regress fine-grained pixel-wise Gaussians. Experiments on several benchmark datasets demonstrate that our method outperforms state-of-the-art methods in novel view synthesis and cross-dataset generalization. Our code is available at https://github.com/iSEE-Laboratory/RoGSplat.

Related papers

• MonoGSDF: Exploring Monocular Geometric Cues for Gaussian Splatting-Guided Implicit Surface Reconstruction [84.07233691641193]
  We introduce MonoGSDF, a novel method that couples primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction. To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization. Experiments on real-world datasets outperforms prior methods while maintaining efficiency.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views [67.34073368933814]
  We propose a generalizable Gaussian Splatting approach for high-resolution image rendering under a sparse-view camera setting. We train our Gaussian parameter regression module on human-only data or human-scene data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. Experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.
  arXiv  Detail & Related papers  (2024-11-18T08:18:44Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Images [20.089890859122168]
  We introduce UniGS, a novel 3D Gaussian reconstruction and novel view synthesis model. UniGS predicts a high-fidelity representation of 3D Gaussians from arbitrary number of posed sparse-view images.
  arXiv  Detail & Related papers  (2024-10-17T03:48:02Z)
• EVA-Gaussian: 3D Gaussian-based Real-time Human Novel View Synthesis under Diverse Multi-view Camera Settings [11.248908608011941]
  3D Gaussian Splatting methods have demonstrated exceptional capability in real-time novel view synthesis for human models.<n>We propose a novel pipeline named EVA-Gaussian for 3D human novel view synthesis across diverse multi-view camera settings.
  arXiv  Detail & Related papers  (2024-10-02T11:23:08Z)
• Generalizable Human Gaussians from Single-View Image [52.100234836129786]
  We introduce a single-view generalizable Human Gaussian Model (HGM) Our approach uses a ControlNet to refine rendered back-view images from coarse predicted human Gaussians. To mitigate the potential generation of unrealistic human poses and shapes, we incorporate human priors from the SMPL-X model as a dual branch.
  arXiv  Detail & Related papers  (2024-06-10T06:38:11Z)
• GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis [70.24111297192057]
  We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting.
  arXiv  Detail & Related papers  (2023-12-04T18:59:55Z)
• Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images [47.14713579719103]
  We introduce a dense depth map as a geometry guide to mitigate overfitting. The adjusted depth aids in the color-based optimization of 3D Gaussian splatting. We verify the proposed method on the NeRF-LLFF dataset with varying numbers of few images.
  arXiv  Detail & Related papers  (2023-11-22T13:53:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.