Iterative approach to reconstructing neural disparity fields from light-field data

• URL: http://arxiv.org/abs/2407.15380v2
• Date: Mon, 30 Jun 2025 02:27:07 GMT
• Title: Iterative approach to reconstructing neural disparity fields from light-field data
• Abstract summary: This study proposes a neural disparity field (NDF) that establishes an implicit, continuous representation of scene disparity.<n>NDF enables seamless and precise characterization of disparity variations in three-dimensional scenes.
• Score: 7.846043927661275
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study proposes a neural disparity field (NDF) that establishes an implicit, continuous representation of scene disparity based on a neural field and an iterative approach to address the inverse problem of NDF reconstruction from light-field data. NDF enables seamless and precise characterization of disparity variations in three-dimensional scenes and can discretize disparity at any arbitrary resolution, overcoming the limitations of traditional disparity maps that are prone to sampling errors and interpolation inaccuracies. The proposed NDF network architecture utilizes hash encoding combined with multilayer perceptrons to capture detailed disparities in texture levels, thereby enhancing its ability to represent the geometric information of complex scenes. By leveraging the spatial-angular consistency inherent in light-field data, a differentiable forward model to generate a central view image from the light-field data is developed. Based on the forward model, an optimization scheme for the inverse problem of NDF reconstruction using differentiable propagation operators is established. Furthermore, an iterative solution method is adopted to reconstruct the NDF in the optimization scheme, which does not require training datasets and applies to light-field data captured by various acquisition methods. Experimental results demonstrate that high-quality NDF can be reconstructed from light-field data using the proposed method. High-resolution disparity can be effectively recovered by NDF, demonstrating its capability for the implicit, continuous representation of scene disparities.

Related papers

• Fast Implicit Neural Light Field Representation via Geometric Decomposition and Multi-Resolution Low-Rank Features [5.088590848084556]
  Implicit neural representations provide a compact and continuous way to reconstruct dense light fields from sampled ray coordinates.<n>Fast light field reconstruction remains challenging because a light field is a high-dimensional signal with strong spatial-angular redundancy and structured disparity variations.<n>This paper proposes a fast implicit light field representation based on geometric decomposition and multi-resolution low-rank features.
  arXiv  Detail & Related papers  (2026-08-14T04:44:14Z)
• Physics-Driven Neural Network for Solving Electromagnetic Inverse Scattering Problems [7.564377962527698]
  A new solution is proposed for inverse scattering problems (ISPs) using a physics-driven neural network (PDNN)<n>PDNN is trained only requiring the input of collected scattered fields and the computation of scattered fields corresponding to predicted solutions.<n>Results demonstrate that the proposed scheme has high reconstruction accuracy and strong stability, even when dealing with composite lossy scatterers.
  arXiv  Detail & Related papers  (2025-07-22T08:04:50Z)
• Geometric Neural Process Fields [58.77241763774756]
  Geometric Neural Process Fields (G-NPF) is a probabilistic framework for neural radiance fields that explicitly captures uncertainty. Building on these bases, we design a hierarchical latent variable model, allowing G-NPF to integrate structural information across multiple spatial levels. Experiments on novel-view synthesis for 3D scenes, as well as 2D image and 1D signal regression, demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2025-02-04T14:17:18Z)
• Scalable and High-Quality Neural Implicit Representation for 3D Reconstruction [34.32994179318829]
  We propose a versatile, scalable and high-quality neural implicit representation to address these issues. We model the object or scene as a fusion of multiple independent local neural SDFs with overlapping regions. Thanks to the independent representation of each local region, our approach can not only achieve high-fidelity surface reconstruction, but also enable scalable scene reconstruction.
  arXiv  Detail & Related papers  (2025-01-15T04:56:26Z)
• Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems [15.603880588503355]
  We introduce a novel depth estimation technique for multi-frame structured light setups using neural implicit representations of 3D space. Our approach employs a neural signed distance field (SDF), trained through self-supervised differentiable rendering.
  arXiv  Detail & Related papers  (2024-05-20T13:24:35Z)
• Light Field Diffusion for Single-View Novel View Synthesis [32.59286750410843]
  Single-view novel view synthesis (NVS) is important but challenging in computer vision. Recent advancements in NVS have leveraged Denoising Diffusion Probabilistic Models (DDPMs) for their exceptional ability to produce high-fidelity images. We present Light Field Diffusion (LFD), a novel conditional diffusion-based approach that transcends the conventional reliance on camera pose matrices.
  arXiv  Detail & Related papers  (2023-09-20T03:27:06Z)
• Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail [54.03399077258403]
  We present LoD-NeuS, an efficient neural representation for high-frequency geometry detail recovery and anti-aliased novel view rendering. Our representation aggregates space features from a multi-convolved featurization within a conical frustum along a ray.
  arXiv  Detail & Related papers  (2023-09-19T05:44:00Z)
• Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization [73.3605319281966]
  We propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.
  arXiv  Detail & Related papers  (2023-09-04T10:42:56Z)
• FFEINR: Flow Feature-Enhanced Implicit Neural Representation for Spatio-temporal Super-Resolution [4.577685231084759]
  This paper proposes a Feature-Enhanced Neural Implicit Representation (FFEINR) for super-resolution of flow field data. It can take full advantage of the implicit neural representation in terms of model structure and sampling resolution. The training process of FFEINR is facilitated by introducing feature enhancements for the input layer.
  arXiv  Detail & Related papers  (2023-08-24T02:28:18Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• RecFNO: a resolution-invariant flow and heat field reconstruction method from sparse observations via Fourier neural operator [8.986743262828009]
  We propose an end-to-end physical field reconstruction method with both excellent performance and mesh transferability named RecFNO. The proposed method aims to learn the mapping from sparse observations to flow and heat field in infinite-dimensional space. The experiments conducted on fluid mechanics and thermology problems show that the proposed method outperforms existing POD-based and CNN-based methods in most cases.
  arXiv  Detail & Related papers  (2023-02-20T07:20:22Z)
• NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies [87.06532943371575]
  We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
  arXiv  Detail & Related papers  (2022-11-25T15:21:45Z)
• IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis [90.03590032170169]
  We present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method. Our experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results.
  arXiv  Detail & Related papers  (2022-10-02T22:45:11Z)
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo [97.07453889070574]
  We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors. We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
  arXiv  Detail & Related papers  (2021-09-02T17:54:31Z)
• Model-inspired Deep Learning for Light-Field Microscopy with Application to Neuron Localization [27.247818386065894]
  We propose a model-inspired deep learning approach to perform fast and robust 3D localization of sources using light-field microscopy images. This is achieved by developing a deep network that efficiently solves a convolutional sparse coding problem. Experiments on localization of mammalian neurons from light-fields show that the proposed approach simultaneously provides enhanced performance, interpretability and efficiency.
  arXiv  Detail & Related papers  (2021-03-10T16:24:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.