Mip-Splatting: Alias-free 3D Gaussian Splatting
- URL: http://arxiv.org/abs/2311.16493v1
- Date: Mon, 27 Nov 2023 13:03:09 GMT
- Title: Mip-Splatting: Alias-free 3D Gaussian Splatting
- Authors: Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger
- Abstract summary: 3D Gaussian Splatting has demonstrated impressive novel view synthesis results, reaching high fidelity and efficiency.
Strong artifacts can be observed when changing the sampling rate, eg, by changing focal length or camera distance.
We find that the source for this phenomenon can be attributed to the lack of 3D frequency constraints and the usage of a 2D dilation filter.
- Score: 52.366815964832426
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, 3D Gaussian Splatting has demonstrated impressive novel view
synthesis results, reaching high fidelity and efficiency. However, strong
artifacts can be observed when changing the sampling rate, \eg, by changing
focal length or camera distance. We find that the source for this phenomenon
can be attributed to the lack of 3D frequency constraints and the usage of a 2D
dilation filter. To address this problem, we introduce a 3D smoothing filter
which constrains the size of the 3D Gaussian primitives based on the maximal
sampling frequency induced by the input views, eliminating high-frequency
artifacts when zooming in. Moreover, replacing 2D dilation with a 2D Mip
filter, which simulates a 2D box filter, effectively mitigates aliasing and
dilation issues. Our evaluation, including scenarios such a training on
single-scale images and testing on multiple scales, validates the effectiveness
of our approach.
Related papers
- MVG-Splatting: Multi-View Guided Gaussian Splatting with Adaptive Quantile-Based Geometric Consistency Densification [8.099621725105857]
We introduce MVG-Splatting, a solution guided by Multi-View considerations.
We propose an adaptive quantile-based method that dynamically determines the level of additional densification.
This approach significantly enhances the overall fidelity and accuracy of the 3D reconstruction process.
arXiv Detail & Related papers (2024-07-16T15:24:01Z) - RecGS: Removing Water Caustic with Recurrent Gaussian Splatting [13.87415686123919]
Water caustics are commonly observed in seafloor imaging data from shallow-water areas.
Traditional methods that remove caustic patterns from images often rely on 2D filtering or pre-training on an annotated dataset.
We present a novel method Recurrent Gaussian Splatting (RecGS), which takes advantage of today's photorealistic 3D reconstruction technology.
arXiv Detail & Related papers (2024-07-14T20:24:44Z) - OccGaussian: 3D Gaussian Splatting for Occluded Human Rendering [55.50438181721271]
Previous method utilizing NeRF for surface rendering to recover the occluded areas requires more than one day to train and several seconds to render occluded areas.
We propose OccGaussian based on 3D Gaussian Splatting, which can be trained within 6 minutes and produces high-quality human renderings up to 160 FPS with occluded input.
arXiv Detail & Related papers (2024-04-12T13:00:06Z) - SpatialTracker: Tracking Any 2D Pixels in 3D Space [71.58016288648447]
We propose to estimate point trajectories in 3D space to mitigate the issues caused by image projection.
Our method, named SpatialTracker, lifts 2D pixels to 3D using monocular depth estimators.
Tracking in 3D allows us to leverage as-rigid-as-possible (ARAP) constraints while simultaneously learning a rigidity embedding that clusters pixels into different rigid parts.
arXiv Detail & Related papers (2024-04-05T17:59:25Z) - Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration [49.004898985671815]
3DGS is not alias-free, and its rendering at varying resolutions could produce severe blurring or jaggies.
This is because 3DGS treats each pixel as an isolated, single point rather than as an area, causing insensitivity to changes in the footprints of pixels.
We introduce this approximation in the two-dimensional pixel shading, and present Analytic-Splatting, which analytically approximates the Gaussian integral within the 2D-pixel window area.
arXiv Detail & Related papers (2024-03-17T02:06:03Z) - Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian
Splatting [57.80942520483354]
3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components.
We introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics.
Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality.
arXiv Detail & Related papers (2024-02-24T17:22:15Z) - Improving Robustness for Joint Optimization of Camera Poses and
Decomposed Low-Rank Tensorial Radiance Fields [26.4340697184666]
We propose an algorithm that allows joint refinement of camera pose and scene geometry represented by decomposed low-rank tensor.
We also propose techniques of smoothed 2D supervision, randomly scaled kernel parameters, and edge-guided loss mask.
arXiv Detail & Related papers (2024-02-20T18:59:02Z) - StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering [42.91830228828405]
We present a novel hierarchicalization approach that culls splats with minimal processing overhead.
Our approach is only 4% slower on average than the original Gaussian Splatting.
rendering performance is nearly doubled, making our approach 1.6x faster than the original Gaussian Splatting.
arXiv Detail & Related papers (2024-02-01T11:46:44Z) - Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image.
Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space.
We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step.
This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
arXiv Detail & Related papers (2020-08-31T17:10:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.