WavePlanes: A compact Wavelet representation for Dynamic Neural Radiance Fields

• URL: http://arxiv.org/abs/2312.02218v3
• Date: Wed, 8 May 2024 13:24:32 GMT
• Title: WavePlanes: A compact Wavelet representation for Dynamic Neural Radiance Fields
• Authors: Adrian Azzarelli, Nantheera Anantrasirichai, David R Bull,
• Abstract summary: This paper presents WavePlanes, a fast and more compact explicit model. We propose a multi-scale space and space-time feature plane representation using N-level 2-D wavelet coefficients. Exploiting the sparsity of wavelet coefficients, we compress the model using a Hash Map.
• Score: 9.158626732325915
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dynamic Neural Radiance Fields (Dynamic NeRF) enhance NeRF technology to model moving scenes. However, they are resource intensive and challenging to compress. To address these issues, this paper presents WavePlanes, a fast and more compact explicit model. We propose a multi-scale space and space-time feature plane representation using N-level 2-D wavelet coefficients. The inverse discrete wavelet transform reconstructs feature signals at varying detail, which are linearly decoded to approximate the color and density of volumes in a 4-D grid. Exploiting the sparsity of wavelet coefficients, we compress the model using a Hash Map containing only non-zero coefficients and their locations on each plane. Compared to the state-of-the-art (SotA) plane-based models, WavePlanes is up to 15x smaller while being less resource demanding and competitive in performance and training time. Compared to other small SotA models WavePlanes preserves details better without requiring custom CUDA code or high performance computing resources. Our code is available at: https://github.com/azzarelli/waveplanes/

Related papers

• Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs [5.583906047971048]
  3D Splatting is a recognized method for 3D scene representation, known for its high rendering quality and speed. We introduce an efficient compression technique that significantly reduces storage overhead by using compact representation. Experimental results demonstrate that our method outperforms existing methods in data compactness while maintaining high rendering quality.
  arXiv  Detail & Related papers  (2025-01-06T21:37:30Z)
• DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction [26.39519157164198]
  We present DaRePlane, a novel representation approach that captures dynamics from six different directions. DaRePlane yields state-of-the-art performance in novel view synthesis for various complex dynamic scenes.
  arXiv  Detail & Related papers  (2024-10-18T04:19:10Z)
• WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration [68.25711405944239]
  Deep image registration has demonstrated exceptional accuracy and fast inference. Recent advances have adopted either multiple cascades or pyramid architectures to estimate dense deformation fields in a coarse-to-fine manner. We introduce a model-driven WiNet that incrementally estimates scale-wise wavelet coefficients for the displacement/velocity field across various scales.
  arXiv  Detail & Related papers  (2024-07-18T11:51:01Z)
• SAGS: Structure-Aware 3D Gaussian Splatting [53.6730827668389]
  We propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene. SAGS reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets.
  arXiv  Detail & Related papers  (2024-04-29T23:26:30Z)
• Concise Plane Arrangements for Low-Poly Surface and Volume Modelling [9.254047358707016]
  We introduce two key novelties that enable the construction of plane arrangements for complex objects and entire scenes. We show that our approach leads to state-of-the-art results by comparing it to learning-based and traditional approaches on various different datasets.
  arXiv  Detail & Related papers  (2024-04-09T09:27:54Z)
• Make-A-Shape: a Ten-Million-scale 3D Shape Model [52.701745578415796]
  This paper introduces Make-A-Shape, a new 3D generative model designed for efficient training on a vast scale. We first innovate a wavelet-tree representation to compactly encode shapes by formulating the subband coefficient filtering scheme. We derive the subband adaptive training strategy to train our model to effectively learn to generate coarse and detail wavelet coefficients.
  arXiv  Detail & Related papers  (2024-01-20T00:21:58Z)
• HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces [71.1071688018433]
  Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render. We propose a method, HybridNeRF, that leverages the strengths of both representations by rendering most objects as surfaces. We improve error rates by 15-30% while achieving real-time framerates (at least 36 FPS) for virtual-reality resolutions (2Kx2K)
  arXiv  Detail & Related papers  (2023-12-05T22:04:49Z)
• Dynamic Frame Interpolation in Wavelet Domain [57.25341639095404]
  Video frame is an important low-level computation vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.
  arXiv  Detail & Related papers  (2023-09-07T06:41:15Z)
• Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation [54.09274684734721]
  We present a new approach for 3D shape generation, inversion, and manipulation, through a direct generative modeling on a continuous implicit representation in wavelet domain. Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets. We may jointly train an encoder network to learn a latent space for inverting shapes, allowing us to enable a rich variety of whole-shape and region-aware shape manipulations.
  arXiv  Detail & Related papers  (2023-02-01T02:47:53Z)
• aiWave: Volumetric Image Compression with 3-D Trained Affine Wavelet-like Transform [43.984890290691695]
  Most commonly used volumetric image compression methods are based on wavelet transform, such as JP3D. In this paper, we first design a 3-D trained wavelet-like transform to enable signal-dependent and non-separable transform. Then, an affine wavelet basis is introduced to capture the various local correlations in different regions of volumetric images.
  arXiv  Detail & Related papers  (2022-03-11T10:02:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.