Mip-Splatting: Alias-free 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2311.16493v1
• Date: Mon, 27 Nov 2023 13:03:09 GMT
• Title: Mip-Splatting: Alias-free 3D Gaussian Splatting
• Authors: Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger
• Abstract summary: 3D Gaussian Splatting has demonstrated impressive novel view synthesis results, reaching high fidelity and efficiency. Strong artifacts can be observed when changing the sampling rate, eg, by changing focal length or camera distance. We find that the source for this phenomenon can be attributed to the lack of 3D frequency constraints and the usage of a 2D dilation filter.
• Score: 52.366815964832426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, 3D Gaussian Splatting has demonstrated impressive novel view synthesis results, reaching high fidelity and efficiency. However, strong artifacts can be observed when changing the sampling rate, \eg, by changing focal length or camera distance. We find that the source for this phenomenon can be attributed to the lack of 3D frequency constraints and the usage of a 2D dilation filter. To address this problem, we introduce a 3D smoothing filter which constrains the size of the 3D Gaussian primitives based on the maximal sampling frequency induced by the input views, eliminating high-frequency artifacts when zooming in. Moreover, replacing 2D dilation with a 2D Mip filter, which simulates a 2D box filter, effectively mitigates aliasing and dilation issues. Our evaluation, including scenarios such a training on single-scale images and testing on multiple scales, validates the effectiveness of our approach.

Related papers

• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis [72.53316783628803]
  We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recentization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering. We show that our method is more accurate with blending issues than 3DGS and follow-up work on view rendering.
  arXiv  Detail & Related papers  (2024-10-02T17:59:09Z)
• 3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for 3D Object Detection [12.14595005884025]
  This paper introduces 3DGS into 3DOD for the first time, identifying two main challenges. We propose an elegant and efficient solution by incorporating 2D Boundary Guidance. We also propose a Box-Focused Sampling strategy using 2D boxes to generate object probability distribution in 3D spaces.
  arXiv  Detail & Related papers  (2024-10-02T15:15:52Z)
• Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy [14.320240635262756]
  3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies.
  arXiv  Detail & Related papers  (2024-09-19T13:38:04Z)
• RecGS: Removing Water Caustic with Recurrent Gaussian Splatting [13.87415686123919]
  Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or pre-training on an annotated dataset. We present a novel method Recurrent Gaussian Splatting (RecGS), which takes advantage of today's photorealistic 3D reconstruction technology.
  arXiv  Detail & Related papers  (2024-07-14T20:24:44Z)
• Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration [49.004898985671815]
  3DGS is not alias-free, and its rendering at varying resolutions could produce severe blurring or jaggies. This is because 3DGS treats each pixel as an isolated, single point rather than as an area, causing insensitivity to changes in the footprints of pixels. We introduce this approximation in the two-dimensional pixel shading, and present Analytic-Splatting, which analytically approximates the Gaussian integral within the 2D-pixel window area.
  arXiv  Detail & Related papers  (2024-03-17T02:06:03Z)
• Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting [55.71424195454963]
  Spec-Gaussian is an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces.
  arXiv  Detail & Related papers  (2024-02-24T17:22:15Z)
• StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering [42.91830228828405]
  We present a novel hierarchicalization approach that culls splats with minimal processing overhead. Our approach is only 4% slower on average than the original Gaussian Splatting. rendering performance is nearly doubled, making our approach 1.6x faster than the original Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-02-01T11:46:44Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.