FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization

• URL: http://arxiv.org/abs/2403.06908v2
• Date: Mon, 8 Apr 2024 16:16:56 GMT
• Title: FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization
• Authors: Jiahui Zhang, Fangneng Zhan, Muyu Xu, Shijian Lu, Eric Xing,
• Abstract summary: We develop a progressive frequency regularization technique to tackle the over-reconstruction issue within the frequency space. FreGS achieves superior novel view synthesis and outperforms the state-of-the-art consistently.
• Score: 67.47895278233717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian splatting has achieved very impressive performance in real-time novel view synthesis. However, it often suffers from over-reconstruction during Gaussian densification where high-variance image regions are covered by a few large Gaussians only, leading to blur and artifacts in the rendered images. We design a progressive frequency regularization (FreGS) technique to tackle the over-reconstruction issue within the frequency space. Specifically, FreGS performs coarse-to-fine Gaussian densification by exploiting low-to-high frequency components that can be easily extracted with low-pass and high-pass filters in the Fourier space. By minimizing the discrepancy between the frequency spectrum of the rendered image and the corresponding ground truth, it achieves high-quality Gaussian densification and alleviates the over-reconstruction of Gaussian splatting effectively. Experiments over multiple widely adopted benchmarks (e.g., Mip-NeRF360, Tanks-and-Temples and Deep Blending) show that FreGS achieves superior novel view synthesis and outperforms the state-of-the-art consistently.

Related papers

• Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy [14.320240635262756]
  3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies.
  arXiv  Detail & Related papers  (2024-09-19T13:38:04Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering [112.16239342037714]
  GES (Generalized Exponential Splatting) is a novel representation that employs Generalized Exponential Function (GEF) to model 3D scenes. With the aid of a frequency-modulated loss, GES achieves competitive performance in novel-view synthesis benchmarks.
  arXiv  Detail & Related papers  (2024-02-15T17:32:50Z)
• LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS [55.85673901231235]
  We introduce LightGaussian, a method for transforming 3D Gaussians into a more compact format. Inspired by Network Pruning, LightGaussian identifies Gaussians with minimal global significance on scene reconstruction. LightGaussian achieves an average 15x compression rate while boosting FPS from 144 to 237 within the 3D-GS framework.
  arXiv  Detail & Related papers  (2023-11-28T21:39:20Z)
• GaussianDiffusion: 3D Gaussian Splatting for Denoising Diffusion Probabilistic Models with Structured Noise [0.0]
  This paper introduces a novel text to 3D content generation framework based on Gaussian splatting. The challenge of achieving multi-view consistency in 3D generation significantly impedes modeling complexity and accuracy.
  arXiv  Detail & Related papers  (2023-11-19T04:26:16Z)
• Guided Frequency Loss for Image Restoration [0.9821874476902969]
  We propose the Guided Frequency Loss (GFL), which helps the model to learn in a balanced way the image's frequency content alongside the spatial content. It aggregates three major components that work in parallel to enhance learning efficiency. We found that the GFL loss improved the PSNR metric in most implemented experiments.
  arXiv  Detail & Related papers  (2023-09-27T10:37:51Z)
• WaveNeRF: Wavelet-based Generalizable Neural Radiance Fields [149.2296890464997]
  We design WaveNeRF, which integrates wavelet frequency decomposition into MVS and NeRF. WaveNeRF achieves superior generalizable radiance field modeling when only given three images as input.
  arXiv  Detail & Related papers  (2023-08-09T09:24:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.