GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object
with Gaussian Splatting
- URL: http://arxiv.org/abs/2402.10259v2
- Date: Tue, 20 Feb 2024 11:19:46 GMT
- Title: GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object
with Gaussian Splatting
- Authors: Chen Yang and Sikuang Li and Jiemin Fang and Ruofan Liang and Lingxi
Xie and Xiaopeng Zhang and Wei Shen and Qi Tian
- Abstract summary: Reconstructing and rendering 3D objects from highly sparse views is of critical importance for promoting applications of 3D vision techniques.
We present a framework to represent and render the 3D object with Gaussian splatting, that achieves high rendering quality with only 4 input images.
Our method is evaluated on several challenging datasets, including MipNeRF360, OmniObject3D, and OpenIllumination.
- Score: 85.83922043049235
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reconstructing and rendering 3D objects from highly sparse views is of
critical importance for promoting applications of 3D vision techniques and
improving user experience. However, images from sparse views only contain very
limited 3D information, leading to two significant challenges: 1) Difficulty in
building multi-view consistency as images for matching are too few; 2)
Partially omitted or highly compressed object information as view coverage is
insufficient. To tackle these challenges, we propose GaussianObject, a
framework to represent and render the 3D object with Gaussian splatting, that
achieves high rendering quality with only 4 input images. We first introduce
techniques of visual hull and floater elimination which explicitly inject
structure priors into the initial optimization process for helping build
multi-view consistency, yielding a coarse 3D Gaussian representation. Then we
construct a Gaussian repair model based on diffusion models to supplement the
omitted object information, where Gaussians are further refined. We design a
self-generating strategy to obtain image pairs for training the repair model.
Our GaussianObject is evaluated on several challenging datasets, including
MipNeRF360, OmniObject3D, and OpenIllumination, achieving strong reconstruction
results from only 4 views and significantly outperforming previous
state-of-the-art methods.
Related papers
- GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction [52.04103235260539]
We present a diffusion model approach based on Gaussian Splatting representation for 3D object reconstruction from a single view.
The model learns to generate 3D objects represented by sets of GS ellipsoids.
The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views.
arXiv Detail & Related papers (2024-07-05T03:43:08Z) - Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting [33.01987451251659]
3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction.
Despite its potential, 3DGS encounters challenges, including needle-like artifacts, suboptimal geometries, and inaccurate normals.
We introduce effective rank as a regularization, which constrains the structure of the Gaussians.
arXiv Detail & Related papers (2024-06-17T15:51:59Z) - MVGamba: Unify 3D Content Generation as State Space Sequence Modeling [150.80564081817786]
We introduce MVGamba, a general and lightweight Gaussian reconstruction model featuring a multi-view Gaussian reconstructor.
With off-the-detail multi-view diffusion models integrated, MVGamba unifies 3D generation tasks from a single image, sparse images, or text prompts.
Experiments demonstrate that MVGamba outperforms state-of-the-art baselines in all 3D content generation scenarios with approximately only $0.1times$ of the model size.
arXiv Detail & Related papers (2024-06-10T15:26:48Z) - CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians [18.42203035154126]
We introduce a structured Gaussian representation that can be controlled in 2D image space.
We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization.
We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes.
arXiv Detail & Related papers (2024-03-28T15:27:13Z) - FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model [81.03553265684184]
We introduce FDGaussian, a novel two-stage framework for single-image 3D reconstruction.
Recent methods typically utilize pre-trained 2D diffusion models to generate plausible novel views from the input image.
We demonstrate that FDGaussian generates images with high consistency across different views and reconstructs high-quality 3D objects.
arXiv Detail & Related papers (2024-03-15T12:24:36Z) - AGG: Amortized Generative 3D Gaussians for Single Image to 3D [108.38567665695027]
We introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image.
AGG decomposes the generation of 3D Gaussian locations and other appearance attributes for joint optimization.
We propose a cascaded pipeline that first generates a coarse representation of the 3D data and later upsamples it with a 3D Gaussian super-resolution module.
arXiv Detail & Related papers (2024-01-08T18:56:33Z) - Sparse-view CT Reconstruction with 3D Gaussian Volumetric Representation [13.667470059238607]
Sparse-view CT is a promising strategy for reducing the radiation dose of traditional CT scans.
Recently, 3D Gaussian has been applied to model complex natural scenes.
We investigate their potential for sparse-view CT reconstruction.
arXiv Detail & Related papers (2023-12-25T09:47:33Z) - Secrets of 3D Implicit Object Shape Reconstruction in the Wild [92.5554695397653]
Reconstructing high-fidelity 3D objects from sparse, partial observation is crucial for various applications in computer vision, robotics, and graphics.
Recent neural implicit modeling methods show promising results on synthetic or dense datasets.
But, they perform poorly on real-world data that is sparse and noisy.
This paper analyzes the root cause of such deficient performance of a popular neural implicit model.
arXiv Detail & Related papers (2021-01-18T03:24:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.