Incorporating Season and Solar Specificity into Renderings made by a NeRF Architecture using Satellite Images

• URL: http://arxiv.org/abs/2308.01262v2
• Date: Fri, 15 Dec 2023 16:33:17 GMT
• Title: Incorporating Season and Solar Specificity into Renderings made by a NeRF Architecture using Satellite Images
• Abstract summary: We show how one can make the renderings season-specific using a NeRF-based framework. NeRF can render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a result of Shadow NeRF and Sat-NeRF, it is possible to take the solar angle into account in a NeRF-based framework for rendering a scene from a novel viewpoint using satellite images for training. Our work extends those contributions and shows how one can make the renderings season-specific. Our main challenge was creating a Neural Radiance Field (NeRF) that could render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We teach our network to render seasonal features by introducing one more input variable -- time of the year. However, the small training datasets typical of satellite imagery can introduce ambiguities in cases where shadows are present in the same location for every image of a particular season. We add additional terms to the loss function to discourage the network from using seasonal features for accounting for shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite. This evaluation includes tests measuring the ability of our framework to accurately render novel views, generate height maps, predict shadows, and specify seasonal features independently from shadows. Our ablation studies justify the choices made for network design parameters.

Related papers

• DeepShade: Enable Shade Simulation by Text-conditioned Image Generation [25.13463236406671]
  Current routing systems fail to incorporate shade information due to the difficulty of estimating shades directly from satellite imagery.<n>We build an extensive dataset covering diverse longitude-latitude regions, varying levels of building density, and different urban layouts.<n>We propose the DeepShade, a diffusion-based model designed to learn and synthesize shade variations over time.
  arXiv  Detail & Related papers  (2025-07-16T10:19:12Z)
• Exploring Seasonal Variability in the Context of Neural Radiance Fields for 3D Reconstruction on Satellite Imagery [0.16492989697868887]
  The study explores how Sat-NeRF, a novel approach in computer vision, performs in predicting seasonal variations across different months. We propose Planet-NeRF, an extension to Sat-NeRF capable of incorporating seasonal variability through a set of month embedding vectors.
  arXiv  Detail & Related papers  (2024-11-05T10:16:14Z)
• Boosting Point Clouds Rendering via Radiance Mapping [49.24193509772339]
  We focus on boosting the image quality of point clouds rendering with a compact model design. We simplify the NeRF representation to a spatial mapping function which only requires single evaluation per pixel. Our method achieves the state-of-the-art rendering on point clouds, outperforming prior works by notable margins.
  arXiv  Detail & Related papers  (2022-10-27T01:25:57Z)
• Towards Learning Neural Representations from Shadows [11.60149896896201]
  We present a method that learns neural scene representations from only shadows present in the scene. Our framework is highly generalizable and can work alongside existing 3D reconstruction techniques.
  arXiv  Detail & Related papers  (2022-03-29T23:13:41Z)
• Sat-NeRF: Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras [10.269997499911668]
  We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogram in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models. We evaluate Sat-NeRF using WorldView-3 images from different locations and stress the advantages of applying a bundle adjustment to the satellite camera models prior to training.
  arXiv  Detail & Related papers  (2022-03-16T19:18:46Z)
• NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild [80.09093712055682]
  We introduce a surface analog of implicit models called Neural Reflectance Surfaces (NeRS) NeRS learns a neural shape representation of a closed surface that is diffeomorphic to a sphere, guaranteeing water-tight reconstructions. We demonstrate that surface-based neural reconstructions enable learning from such data, outperforming volumetric neural rendering-based reconstructions.
  arXiv  Detail & Related papers  (2021-10-14T17:59:58Z)
• Vision-based Neural Scene Representations for Spacecraft [1.0323063834827415]
  In advanced mission concepts, spacecraft need to internally model the pose and shape of nearby orbiting objects. Recent works in neural scene representations show promising results for inferring generic three-dimensional scenes from optical images. We compare and evaluate the potential of NeRF and GRAF to render novel views and extract the 3D shape of two different spacecraft.
  arXiv  Detail & Related papers  (2021-05-11T08:35:05Z)
• Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry [1.370633147306388]
  We present a new generic method for shadow-aware multi-view satellite photogrammetry of Earth Observation scenes. Our proposed method, the Shadow Neural Radiance Field (S-NeRF), follows recent advances in implicit volumetric representation learning. For each scene, we train S-NeRF using very high spatial resolution optical images taken from known viewing angles. The learning requires no labels or shape priors: it is self-supervised by an image reconstruction loss.
  arXiv  Detail & Related papers  (2021-04-20T10:17:34Z)
• Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis [86.38901313994734]
  We present DietNeRF, a 3D neural scene representation estimated from a few images. NeRF learns a continuous volumetric representation of a scene through multi-view consistency. We introduce an auxiliary semantic consistency loss that encourages realistic renderings at novel poses.
  arXiv  Detail & Related papers  (2021-04-01T17:59:31Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)
• NeRF++: Analyzing and Improving Neural Radiance Fields [117.73411181186088]
  Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings. NeRF fits multi-layer perceptrons representing view-invariant opacity and view-dependent color volumes to a set of training images. We address a parametrization issue involved in applying NeRF to 360 captures of objects within large-scale, 3D scenes.
  arXiv  Detail & Related papers  (2020-10-15T03:24:14Z)
• Neural Reflectance Fields for Appearance Acquisition [61.542001266380375]
  We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene. We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light.
  arXiv  Detail & Related papers  (2020-08-09T22:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.