Car-GS: Addressing Reflective and Transparent Surface Challenges in 3D Car Reconstruction

• URL: http://arxiv.org/abs/2501.11020v1
• Date: Sun, 19 Jan 2025 11:49:35 GMT
• Title: Car-GS: Addressing Reflective and Transparent Surface Challenges in 3D Car Reconstruction
• Abstract summary: Car-GS is a novel approach to mitigate the effects of specular highlights and the coupling of RGB and geometry in 3D geometric and shading reconstruction.<n>We show that Car-GS achieves precise reconstruction of car surfaces and significantly outperforms prior methods.
• Score: 15.760408540786035
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D car modeling is crucial for applications in autonomous driving systems, virtual and augmented reality, and gaming. However, due to the distinctive properties of cars, such as highly reflective and transparent surface materials, existing methods often struggle to achieve accurate 3D car reconstruction.To address these limitations, we propose Car-GS, a novel approach designed to mitigate the effects of specular highlights and the coupling of RGB and geometry in 3D geometric and shading reconstruction (3DGS). Our method incorporates three key innovations: First, we introduce view-dependent Gaussian primitives to effectively model surface reflections. Second, we identify the limitations of using a shared opacity parameter for both image rendering and geometric attributes when modeling transparent objects. To overcome this, we assign a learnable geometry-specific opacity to each 2D Gaussian primitive, dedicated solely to rendering depth and normals. Third, we observe that reconstruction errors are most prominent when the camera view is nearly orthogonal to glass surfaces. To address this issue, we develop a quality-aware supervision module that adaptively leverages normal priors from a pre-trained large-scale normal model.Experimental results demonstrate that Car-GS achieves precise reconstruction of car surfaces and significantly outperforms prior methods. The project page is available at https://lcc815.github.io/Car-GS.

Related papers

• SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generation [70.76810765911499]
  SuperCarver is a 3D geometry framework specifically tailored for adding texture-consistent surface details to given coarse meshes. To achieve geometric detail generation, we develop a deterministic prior-guided normal diffusion model fine-tuned on a dataset of paired low-poly and high-poly normal renderings. To optimize mesh structures from potentially imperfect normal map predictions, we design a simple yet effective noise-resistant inverse rendering scheme.
  arXiv  Detail & Related papers  (2025-03-12T14:38:45Z)
• REArtGS: Reconstructing and Generating Articulated Objects via 3D Gaussian Splatting with Geometric and Motion Constraints [48.80178020541189]
  REArtGS is a novel framework that introduces additional geometric and motion constraints to 3D Gaussian primitives. We establish deformable fields for 3D Gaussians constrained by the kinematic structures of articulated objects, achieving unsupervised generation of surface meshes in unseen states.
  arXiv  Detail & Related papers  (2025-03-09T16:05:36Z)
• GlossGau: Efficient Inverse Rendering for Glossy Surface with Anisotropic Spherical Gaussian [4.5442067197725]
  GlossGau is an efficient inverse rendering framework that reconstructs scenes with glossy surfaces while maintaining training and rendering speeds comparable to vanilla 3D-GS. Experiments demonstrate that GlossGau achieves competitive or superior reconstruction on datasets with glossy surfaces.
  arXiv  Detail & Related papers  (2025-02-19T22:20:57Z)
• Gaussian Object Carver: Object-Compositional Gaussian Splatting with surfaces completion [16.379647695019308]
  3D scene reconstruction is a foundational problem in computer vision.<n>We introduce the Gaussian Object Carver (GOC), a novel, efficient, and scalable framework for object-compositional 3D scene reconstruction.<n>GOC leverage 3D Gaussian Splatting (GS), enriched with monocular geometry priors and multi-view geometry regularization, to achieve high-quality and flexible reconstruction.
  arXiv  Detail & Related papers  (2024-12-03T01:34:39Z)
• GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering [83.69136534797686]
  We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
  arXiv  Detail & Related papers  (2024-11-12T01:51:05Z)
• RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Image [30.049602796278133]
  High-quality 3D car assets are essential for various applications, including video games, autonomous driving, and virtual reality. Current 3D generation methods utilizing NeRF or 3D-GS as representations for 3D objects, generate a Lambertian object under fixed lighting. We propose a novel relightable 3D object generative framework that automates the creation of 3D car assets from a single input image.
  arXiv  Detail & Related papers  (2024-10-10T17:54:03Z)
• GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation [65.33726478659304]
  We introduce the Geometry-Aware Large Reconstruction Model (GeoLRM), an approach which can predict high-quality assets with 512k Gaussians and 21 input images in only 11 GB GPU memory. Previous works neglect the inherent sparsity of 3D structure and do not utilize explicit geometric relationships between 3D and 2D images. GeoLRM tackles these issues by incorporating a novel 3D-aware transformer structure that directly processes 3D points and uses deformable cross-attention mechanisms.
  arXiv  Detail & Related papers  (2024-06-21T17:49:31Z)
• Normal-guided Detail-Preserving Neural Implicit Functions for High-Fidelity 3D Surface Reconstruction [6.4279213810512665]
  Current methods for learning neural implicit representations from RGB or RGBD images produce 3D surfaces with missing parts and details. This paper demonstrates that training neural representations with first-order differential properties, i.e. surface normals, leads to highly accurate 3D surface reconstruction.
  arXiv  Detail & Related papers  (2024-06-07T11:48:47Z)
• LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives [44.06145846507639]
  LetsGo is a LiDAR-assisted Gaussian splatting framework for large-scale garage modeling and rendering. We present the GarageWorld dataset, consisting of eight expansive garage scenes with diverse geometric structures. We introduce a novel depth regularizer that effectively eliminates floating artifacts in rendered images.
  arXiv  Detail & Related papers  (2024-04-15T12:50:44Z)
• Ghost on the Shell: An Expressive Representation of General 3D Shapes [97.76840585617907]
  Meshes are appealing since they enable fast physics-based rendering with realistic material and lighting. Recent work on reconstructing and statistically modeling 3D shapes has critiqued meshes as being topologically inflexible. We parameterize open surfaces by defining a manifold signed distance field on watertight surfaces. G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2023-10-23T17:59:52Z)
• Normal Transformer: Extracting Surface Geometry from LiDAR Points Enhanced by Visual Semantics [7.507853813361308]
  We introduce a multi-modal technique that leverages 3D point clouds and 2D colour images obtained from LiDAR and camera sensors for surface normal estimation. We present a novel transformer-based neural network architecture that proficiently fuses visual semantic and 3D geometric information. It has been verified that the proposed model can learn from a simulated 3D environment that mimics a traffic scene.
  arXiv  Detail & Related papers  (2022-11-19T03:55:09Z)
• GAN2X: Non-Lambertian Inverse Rendering of Image GANs [85.76426471872855]
  We present GAN2X, a new method for unsupervised inverse rendering that only uses unpaired images for training. Unlike previous Shape-from-GAN approaches that mainly focus on 3D shapes, we take the first attempt to also recover non-Lambertian material properties by exploiting the pseudo paired data generated by a GAN. Experiments demonstrate that GAN2X can accurately decompose 2D images to 3D shape, albedo, and specular properties for different object categories, and achieves the state-of-the-art performance for unsupervised single-view 3D face reconstruction.
  arXiv  Detail & Related papers  (2022-06-18T16:58:49Z)
• Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection [70.71934539556916]
  We learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection. Specifically, a principled geometry formula with projective modeling of 2D and 3D depth predictions in the monocular 3D object detection network is devised. Our method remarkably improves the detection performance of the state-of-the-art monocular-based method without extra data by 2.80% on the moderate test setting.
  arXiv  Detail & Related papers  (2021-07-29T12:30:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.