ECRF: Entropy-Constrained Neural Radiance Fields Compression with Frequency Domain Optimization

• URL: http://arxiv.org/abs/2311.14208v1
• Date: Thu, 23 Nov 2023 21:23:52 GMT
• Title: ECRF: Entropy-Constrained Neural Radiance Fields Compression with Frequency Domain Optimization
• Authors: Soonbin Lee, Fangwen Shu, Yago Sanchez, Thomas Schierl, Cornelius Hellge
• Abstract summary: Explicit feature-grid based NeRF models have shown promising results in terms of rendering quality and significant speed-up in training. We present a compression model that aims to minimize the entropy in the frequency domain in order to effectively reduce the data size.
• Score: 5.990671011715725
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explicit feature-grid based NeRF models have shown promising results in terms of rendering quality and significant speed-up in training. However, these methods often require a significant amount of data to represent a single scene or object. In this work, we present a compression model that aims to minimize the entropy in the frequency domain in order to effectively reduce the data size. First, we propose using the discrete cosine transform (DCT) on the tensorial radiance fields to compress the feature-grid. This feature-grid is transformed into coefficients, which are then quantized and entropy encoded, following a similar approach to the traditional video coding pipeline. Furthermore, to achieve a higher level of sparsity, we propose using an entropy parameterization technique for the frequency domain, specifically for DCT coefficients of the feature-grid. Since the transformed coefficients are optimized during the training phase, the proposed model does not require any fine-tuning or additional information. Our model only requires a lightweight compression pipeline for encoding and decoding, making it easier to apply volumetric radiance field methods for real-world applications. Experimental results demonstrate that our proposed frequency domain entropy model can achieve superior compression performance across various datasets. The source code will be made publicly available.

Related papers

• Rate-aware Compression for NeRF-based Volumetric Video [21.372568857027748]
  radiance fields (NeRF) have advanced the development of 3D volumetric video technology. Existing solutions compress NeRF representations after the training stage, leading to a separation between representation training and compression. In this paper, we try to directly learn a compact NeRF representation for volumetric video in the training stage based on the proposed rate-aware compression framework.
  arXiv  Detail & Related papers  (2024-11-08T04:29:14Z)
• Neural NeRF Compression [19.853882143024]
  Recent NeRFs utilize feature grids to improve rendering quality and speed. These representations introduce significant storage overhead. This paper presents a novel method for efficiently compressing a grid-based NeRF model.
  arXiv  Detail & Related papers  (2024-06-13T09:12:26Z)
• Frequency-Aware Re-Parameterization for Over-Fitting Based Image Compression [12.725194101094711]
  Over-fitting-based image compression requires weights compactness for compression and fast convergence for practical use. This paper presents a simple re- parameterization method to train CNNs with reduced weights storage and accelerated convergence. The proposed method is verified with extensive experiments of over-fitting-based image restoration on various datasets, achieving up to -46.12% BD-rate on top of HEIF with only 200 iterations.
  arXiv  Detail & Related papers  (2023-10-12T06:32:12Z)
• Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression [63.56922682378755]
  We focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.
  arXiv  Detail & Related papers  (2023-08-17T01:34:51Z)
• Frequency Disentangled Features in Neural Image Compression [13.016298207860974]
  A neural image compression network is governed by how well the entropy model matches the true distribution of the latent code. In this paper, we propose a feature-level frequency disentanglement to help the relaxed scalar quantization achieve lower bit rates. The proposed network not only outperforms hand-engineered codecs, but also neural network-based codecs built on-heavy spatially autoregressive entropy models.
  arXiv  Detail & Related papers  (2023-08-04T14:55:44Z)
• Compression with Bayesian Implicit Neural Representations [16.593537431810237]
  We propose overfitting variational neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding instead of quantizing and entropy coding it. Experiments show that our method achieves strong performance on image and audio compression while retaining simplicity.
  arXiv  Detail & Related papers  (2023-05-30T16:29:52Z)
• LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral Image Generation with Variance Regularization [72.4394510913927]
  Deep learning methods are state-of-the-art for spectral image (SI) computational tasks. GANs enable diverse augmentation by learning and sampling from the data distribution. GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation. We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
  arXiv  Detail & Related papers  (2023-04-29T00:25:02Z)
• Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
  We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time. This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
  arXiv  Detail & Related papers  (2022-11-26T01:56:05Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Rate Distortion Characteristic Modeling for Neural Image Compression [59.25700168404325]
  End-to-end optimization capability offers neural image compression (NIC) superior lossy compression performance. distinct models are required to be trained to reach different points in the rate-distortion (R-D) space. We make efforts to formulate the essential mathematical functions to describe the R-D behavior of NIC using deep network and statistical modeling.
  arXiv  Detail & Related papers  (2021-06-24T12:23:05Z)
• Learning Context-Based Non-local Entropy Modeling for Image Compression [140.64888994506313]
  In this paper, we propose a non-local operation for context modeling by employing the global similarity within the context. The entropy model is further adopted as the rate loss in a joint rate-distortion optimization. Considering that the width of the transforms is essential in training low distortion models, we finally produce a U-Net block in the transforms to increase the width with manageable memory consumption and time complexity.
  arXiv  Detail & Related papers  (2020-05-10T13:28:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.