Focused Specific Objects NeRF

• URL: http://arxiv.org/abs/2308.05970v1
• Date: Fri, 11 Aug 2023 07:07:40 GMT
• Title: Focused Specific Objects NeRF
• Authors: Yuesong Li, Feng Pan, Helong Yan, Xiuli Xin, Xiaoxue Feng
• Abstract summary: This paper utilizes scene semantic priors to make improvements in fast training, allowing the network to focus on the specific targets. The training speed can be increased by 7.78 times with better rendering effect, and small to medium sized targets can be rendered faster. Considering the inherent multi-view consistency and smoothness of NeRF, this paper also studies weak supervision by sparsely sampling negative ray samples.
• Score: 1.1424576927168384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most NeRF-based models are designed for learning the entire scene, and complex scenes can lead to longer learning times and poorer rendering effects. This paper utilizes scene semantic priors to make improvements in fast training, allowing the network to focus on the specific targets and not be affected by complex backgrounds. The training speed can be increased by 7.78 times with better rendering effect, and small to medium sized targets can be rendered faster. In addition, this improvement applies to all NeRF-based models. Considering the inherent multi-view consistency and smoothness of NeRF, this paper also studies weak supervision by sparsely sampling negative ray samples. With this method, training can be further accelerated and rendering quality can be maintained. Finally, this paper extends pixel semantic and color rendering formulas and proposes a new scene editing technique that can achieve unique displays of the specific semantic targets or masking them in rendering. To address the problem of unsupervised regions incorrect inferences in the scene, we also designed a self-supervised loop that combines morphological operations and clustering.

Related papers

• Few-shot NeRF by Adaptive Rendering Loss Regularization [78.50710219013301]
  Novel view synthesis with sparse inputs poses great challenges to Neural Radiance Field (NeRF) Recent works demonstrate that the frequency regularization of Positional rendering can achieve promising results for few-shot NeRF. We propose Adaptive Rendering loss regularization for few-shot NeRF, dubbed AR-NeRF.
  arXiv  Detail & Related papers  (2024-10-23T13:05:26Z)
• IReNe: Instant Recoloring of Neural Radiance Fields [54.94866137102324]
  We introduce IReNe, enabling swift, near real-time color editing in NeRF. We leverage a pre-trained NeRF model and a single training image with user-applied color edits. This adjustment allows the model to generate new scene views, accurately representing the color changes from the training image.
  arXiv  Detail & Related papers  (2024-05-30T09:30:28Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations [34.836151514152746]
  In this work, we investigate combining an autoencoder with a NeRF, in which latent features are rendered and then convolutionally decoded. The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs. We can control the tradeoff between efficiency and image quality by shrinking the AE architecture, achieving over 13 times faster rendering with only a small drop in performance.
  arXiv  Detail & Related papers  (2023-10-27T03:52:08Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction [50.54946139497575]
  We propose NeRFusion, a method that combines the advantages of NeRF and TSDF-based fusion techniques to achieve efficient large-scale reconstruction and photo-realistic rendering. We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
  arXiv  Detail & Related papers  (2022-03-21T18:56:35Z)
• RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs [79.00855490550367]
  We show that NeRF can produce photorealistic renderings of unseen viewpoints when many input views are available. We address this by regularizing the geometry and appearance of patches rendered from unobserved viewpoints. Our model outperforms not only other methods that optimize over a single scene, but also conditional models that are extensively pre-trained on large multi-view datasets.
  arXiv  Detail & Related papers  (2021-12-01T18:59:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.