NeRFlex: Resource-aware Real-time High-quality Rendering of Complex Scenes on Mobile Devices

• URL: http://arxiv.org/abs/2504.03415v1
• Date: Fri, 04 Apr 2025 12:53:33 GMT
• Title: NeRFlex: Resource-aware Real-time High-quality Rendering of Complex Scenes on Mobile Devices
• Authors: Zhe Wang, Yifei Zhu,
• Abstract summary: We present NeRFlex, a real-time rendering framework for complex scenes on mobile devices.<n>NeRFlex integrates mobile NeRF rendering with multi-NeRF representations that decompose a scene into multiple sub-scenes.<n>Experiments on real-world datasets and mobile devices demonstrate that NeRFlex achieves real-time, high-quality rendering.
• Score: 12.392923990003753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Radiance Fields (NeRF) is a cutting-edge neural network-based technique for novel view synthesis in 3D reconstruction. However, its significant computational demands pose challenges for deployment on mobile devices. While mesh-based NeRF solutions have shown potential in achieving real-time rendering on mobile platforms, they often fail to deliver high-quality reconstructions when rendering practical complex scenes. Additionally, the non-negligible memory overhead caused by pre-computed intermediate results complicates their practical application. To overcome these challenges, we present NeRFlex, a resource-aware, high-resolution, real-time rendering framework for complex scenes on mobile devices. NeRFlex integrates mobile NeRF rendering with multi-NeRF representations that decompose a scene into multiple sub-scenes, each represented by an individual NeRF network. Crucially, NeRFlex considers both memory and computation constraints as first-class citizens and redesigns the reconstruction process accordingly. NeRFlex first designs a detail-oriented segmentation module to identify sub-scenes with high-frequency details. For each NeRF network, a lightweight profiler, built on domain knowledge, is used to accurately map configurations to visual quality and memory usage. Based on these insights and the resource constraints on mobile devices, NeRFlex presents a dynamic programming algorithm to efficiently determine configurations for all NeRF representations, despite the NP-hardness of the original decision problem. Extensive experiments on real-world datasets and mobile devices demonstrate that NeRFlex achieves real-time, high-quality rendering on commercial mobile devices.

Related papers

• Efficient Neural Light Fields (ENeLF) for Mobile Devices [0.0]
  This research builds upon the novel network architecture introduced by MobileR2L to produce a model that runs efficiently on mobile devices with lower latency and smaller sizes.
  arXiv  Detail & Related papers  (2024-06-02T02:55:52Z)
• Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs [59.12526668734703]
  We introduce Composable Object Volume NeRF (COV-NeRF), an object-composable NeRF model that is the centerpiece of a real-to-sim pipeline. COV-NeRF extracts objects from real images and composes them into new scenes, generating photorealistic renderings and many types of 2D and 3D supervision.
  arXiv  Detail & Related papers  (2024-03-07T00:00:02Z)
• Hyb-NeRF: A Multiresolution Hybrid Encoding for Neural Radiance Fields [12.335934855851486]
  We present Hyb-NeRF, a novel neural radiance field with a multi-resolution hybrid encoding. We show that Hyb-NeRF achieves faster rendering speed with better rending quality and even a lower memory footprint in comparison to previous methods.
  arXiv  Detail & Related papers  (2023-11-21T10:01:08Z)
• From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm [57.73868344064043]
  We propose NeRFLiX, a general NeRF-agnostic restorer paradigm that learns a degradation-driven inter-viewpoint mixer. We also present NeRFLiX++ with a stronger two-stage NeRF degradation simulator and a faster inter-viewpoint mixer. NeRFLiX++ is capable of restoring photo-realistic ultra-high-resolution outputs from noisy low-resolution NeRF-rendered views.
  arXiv  Detail & Related papers  (2023-06-10T09:19:19Z)
• Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design [17.32288942793313]
  Gen-NeRF is an algorithm-hardware co-design framework dedicated to generalizable NeRF acceleration. On the algorithm side, Gen-NeRF integrates a coarse-then-focus sampling strategy. On the hardware side, Gen-NeRF highlights an accelerator micro-architecture to maximize the data reuse opportunities.
  arXiv  Detail & Related papers  (2023-04-24T06:22:06Z)
• Re-ReND: Real-time Rendering of NeRFs across Devices [56.081995086924216]
  Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines. We find that Re-ReND can achieve over a 2.6-fold increase in rendering speed versus the state-of-the-art without perceptible losses in quality.
  arXiv  Detail & Related papers  (2023-03-15T15:59:41Z)
• MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields [49.68916478541697]
  We develop a Memory-Efficient Incremental Learning algorithm for NeRF (MEIL-NeRF) MEIL-NeRF takes inspiration from NeRF itself in that a neural network can serve as a memory that provides the pixel RGB values, given rays as queries. As a result, MEIL-NeRF demonstrates constant memory consumption and competitive performance.
  arXiv  Detail & Related papers  (2022-12-16T08:04:56Z)
• Real-Time Neural Light Field on Mobile Devices [54.44982318758239]
  We introduce a novel network architecture that runs efficiently on mobile devices with low latency and small size. Our model achieves high-resolution generation while maintaining real-time inference for both synthetic and real-world scenes.
  arXiv  Detail & Related papers  (2022-12-15T18:58:56Z)
• NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction [50.54946139497575]
  We propose NeRFusion, a method that combines the advantages of NeRF and TSDF-based fusion techniques to achieve efficient large-scale reconstruction and photo-realistic rendering. We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
  arXiv  Detail & Related papers  (2022-03-21T18:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.