PEP-GS: Perceptually-Enhanced Precise Structured 3D Gaussians for View-Adaptive Rendering

• URL: http://arxiv.org/abs/2411.05731v1
• Date: Fri, 08 Nov 2024 17:42:02 GMT
• Title: PEP-GS: Perceptually-Enhanced Precise Structured 3D Gaussians for View-Adaptive Rendering
• Authors: Junxi Jin, Xiulai Li, Haiping Huang, Lianjun Liu, Yujie Sun,
• Abstract summary: Recent advances in structured 3D Gaussians for view-adaptive rendering have demonstrated promising results in neural scene representation. We present PEP-GS, a novel framework that enhances structured 3D Gaussians through three key innovations. Our comprehensive evaluation across multiple datasets indicates that, compared to the current state-of-the-art methods, these improvements are particularly evident in challenging scenarios.
• Score: 3.285531771049763
• License:
• Abstract: Recent advances in structured 3D Gaussians for view-adaptive rendering, particularly through methods like Scaffold-GS, have demonstrated promising results in neural scene representation. However, existing approaches still face challenges in perceptual consistency and precise view-dependent effects. We present PEP-GS, a novel framework that enhances structured 3D Gaussians through three key innovations: (1) a Local-Enhanced Multi-head Self-Attention (LEMSA) mechanism that replaces spherical harmonics for more accurate view-dependent color decoding, and (2) Kolmogorov-Arnold Networks (KAN) that optimize Gaussian opacity and covariance functions for enhanced interpretability and splatting precision. (3) a Neural Laplacian Pyramid Decomposition (NLPD) that improves perceptual similarity across views. Our comprehensive evaluation across multiple datasets indicates that, compared to the current state-of-the-art methods, these improvements are particularly evident in challenging scenarios such as view-dependent effects, specular reflections, fine-scale details and false geometry generation.

Related papers

• G2SDF: Surface Reconstruction from Explicit Gaussians with Implicit SDFs [84.07233691641193]
  We introduce G2SDF, a novel approach that integrates a neural implicit Signed Distance Field into the Gaussian Splatting framework. G2SDF achieves superior quality than prior works while maintaining the efficiency of 3DGS.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• MVGS: Multi-view-regulated Gaussian Splatting for Novel View Synthesis [22.80370814838661]
  Recent works in volume rendering, textite.g. NeRF and 3D Gaussian Splatting (3DGS), significantly advance the rendering quality and efficiency. We propose a new 3DGS optimization method embodying four key novel contributions.
  arXiv  Detail & Related papers  (2024-10-02T23:48:31Z)
• Implicit Gaussian Splatting with Efficient Multi-Level Tri-Plane Representation [45.582869951581785]
  Implicit Gaussian Splatting (IGS) is an innovative hybrid model that integrates explicit point clouds with implicit feature embeddings. We introduce a level-based progressive training scheme, which incorporates explicit spatial regularization. Our algorithm can deliver high-quality rendering using only a few MBs, effectively balancing storage efficiency and rendering fidelity.
  arXiv  Detail & Related papers  (2024-08-19T14:34:17Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)
• Mesh-based Gaussian Splatting for Real-time Large-scale Deformation [58.18290393082119]
  It is challenging for users to directly deform or manipulate implicit representations with large deformations in the real-time fashion. We develop a novel GS-based method that enables interactive deformation. Our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate.
  arXiv  Detail & Related papers  (2024-02-07T12:36:54Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting [51.96353586773191]
  We introduce textbfGS-SLAM that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping system. Our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets.
  arXiv  Detail & Related papers  (2023-11-20T12:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.