Compact Real-time Radiance Fields with Neural Codebook

• URL: http://arxiv.org/abs/2305.18163v1
• Date: Mon, 29 May 2023 15:49:20 GMT
• Title: Compact Real-time Radiance Fields with Neural Codebook
• Authors: Lingzhi Li, Zhongshu Wang, Zhen Shen, Li Shen, Ping Tan
• Abstract summary: We present a framework for pursuing compact radiance fields from the perspective of compression methodology. By exploiting intrinsic properties exhibiting in grid models, a non-uniform compression stem is developed to significantly reduce model complexity. Our approach can achieve over 40 $times$ reduction on grid model storage with competitive rendering quality.
• Score: 32.856346090347174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing neural radiance fields with explicit volumetric representations, demonstrated by Plenoxels, has shown remarkable advantages on training and rendering efficiency, while grid-based representations typically induce considerable overhead for storage and transmission. In this work, we present a simple and effective framework for pursuing compact radiance fields from the perspective of compression methodology. By exploiting intrinsic properties exhibiting in grid models, a non-uniform compression stem is developed to significantly reduce model complexity and a novel parameterized module, named Neural Codebook, is introduced for better encoding high-frequency details specific to per-scene models via a fast optimization. Our approach can achieve over 40 $\times$ reduction on grid model storage with competitive rendering quality. In addition, the method can achieve real-time rendering speed with 180 fps, realizing significant advantage on storage cost compared to real-time rendering methods.

Related papers

• RenderFlow: Single-Step Neural Rendering via Flow Matching [17.56739408578129]
  We present a novel end-to-end, deterministic, single-step neural rendering framework, RenderFlow, built upon a flow matching paradigm.<n>Our method significantly accelerates rendering process and enhances both the physical plausibility and overall visual quality of the output.<n>The resulting pipeline achieves near real-time performance with photorealistic rendering quality, effectively bridging the gap between the efficiency of modern generative models and the precision of traditional physically based rendering.
  arXiv  Detail & Related papers  (2026-01-11T14:28:46Z)
• SANR: Scene-Aware Neural Representation for Light Field Image Compression with Rate-Distortion Optimization [54.184486302645716]
  We propose a Scene-Aware Neural Representation framework for light field image compression with end-to-end rate-distortion optimization.<n>For scene awareness, SANR introduces a hierarchical scene modeling block that leverages multi-scale latent codes to capture intrinsic scene structures.<n>Experiment results demonstrate that SANR significantly outperforms state-of-the-art techniques regarding rate-distortion performance with a 65.62% BD-rate saving against HEVC.
  arXiv  Detail & Related papers  (2025-10-17T16:00:43Z)
• Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization [46.18363767705346]
  Deployment of large and complex convex networks on resource-constrained devices significant challenges due to their demands. We introduce novel distillation technique that efficiently compresses model via this model via this paper. Our approach enables performance comparable to original model without requiring any post-processing.
  arXiv  Detail & Related papers  (2024-10-09T06:04:52Z)
• Neural NeRF Compression [19.853882143024]
  Recent NeRFs utilize feature grids to improve rendering quality and speed. These representations introduce significant storage overhead. This paper presents a novel method for efficiently compressing a grid-based NeRF model.
  arXiv  Detail & Related papers  (2024-06-13T09:12:26Z)
• Spatial Annealing Smoothing for Efficient Few-shot Neural Rendering [106.0057551634008]
  We introduce an accurate and efficient few-shot neural rendering method named Spatial Annealing smoothing regularized NeRF (SANeRF) By adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot NeRF methods.
  arXiv  Detail & Related papers  (2024-06-12T02:48:52Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• Adaptive Shells for Efficient Neural Radiance Field Rendering [92.18962730460842]
  We propose a neural radiance formulation that smoothly transitions between- and surface-based rendering. Our approach enables efficient rendering at very high fidelity. We also demonstrate that the extracted envelope enables downstream applications such as animation and simulation.
  arXiv  Detail & Related papers  (2023-11-16T18:58:55Z)
• VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis [51.49008959209671]
  We introduce VoxNeRF, a novel approach that leverages volumetric representations to enhance the quality and efficiency of indoor view synthesis. We employ multi-resolution hash grids to adaptively capture spatial features, effectively managing occlusions and the intricate geometry of indoor scenes. We validate our approach against three public indoor datasets and demonstrate that VoxNeRF outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-11-09T11:32:49Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos [69.22032459870242]
  We present a novel technique, Residual Radiance Field or ReRF, as a highly compact neural representation to achieve real-time free-view rendering on long-duration dynamic scenes. We show such a strategy can handle large motions without sacrificing quality. Based on ReRF, we design a special FVV that achieves three orders of magnitudes compression rate and provides a companion ReRF player to support online streaming of long-duration FVVs of dynamic scenes.
  arXiv  Detail & Related papers  (2023-04-10T08:36:00Z)
• Grid-guided Neural Radiance Fields for Large Urban Scenes [146.06368329445857]
  Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene. We present a new framework that realizes high-fidelity rendering on large urban scenes while being computationally efficient.
  arXiv  Detail & Related papers  (2023-03-24T13:56:45Z)
• Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians [20.68953631807367]
  We propose a novel online framework to learn the spatial-varying density model with a single small neural network. Our framework learns the distribution in a progressive manner and does not need any warm-up phases.
  arXiv  Detail & Related papers  (2023-03-11T05:22:42Z)
• Compressing Volumetric Radiance Fields to 1 MB [19.380248980850727]
  Approximating radiance fields with volumetric grids is one of promising directions for improving NeRF. We introduce a simple yet effective framework, called vector quantized radiance fields (VQRF), for compressing these volume-grid-based radiance fields. In combination with an efficient joint tuning strategy and post-processing, our method can achieve a compression ratio of 100$times$ by reducing the overall model size to 1 MB.
  arXiv  Detail & Related papers  (2022-11-29T17:11:25Z)
• Streaming Radiance Fields for 3D Video Synthesis [32.856346090347174]
  We present an explicit-grid based method for reconstructing streaming radiance fields for novel view synthesis of real world dynamic scenes. Experiments on challenging video sequences demonstrate that our approach is capable of achieving a training speed of 15 seconds per-frame with competitive rendering quality.
  arXiv  Detail & Related papers  (2022-10-26T16:23:02Z)
• Fast Dynamic Radiance Fields with Time-Aware Neural Voxels [106.69049089979433]
  We propose a radiance field framework by representing scenes with time-aware voxel features, named as TiNeuVox. Our framework accelerates the optimization of dynamic radiance fields while maintaining high rendering quality. Our TiNeuVox completes training with only 8 minutes and 8-MB storage cost while showing similar or even better rendering performance than previous dynamic NeRF methods.
  arXiv  Detail & Related papers  (2022-05-30T17:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.