Factor Fields: A Unified Framework for Neural Fields and Beyond

• URL: http://arxiv.org/abs/2302.01226v3
• Date: Thu, 27 Jul 2023 05:29:14 GMT
• Title: Factor Fields: A Unified Framework for Neural Fields and Beyond
• Authors: Anpei Chen, Zexiang Xu, Xinyue Wei, Siyu Tang, Hao Su, Andreas Geiger
• Abstract summary: We present Factor Fields, a novel framework for modeling and representing signals. Our framework accommodates several recent signal representations including NeRF, Plenoxels, EG3D, Instant-NGP, and TensoRF. Our representation achieves better image approximation quality on 2D image regression tasks, higher geometric quality when reconstructing 3D signed distance fields, and higher compactness for radiance field reconstruction tasks.
• Score: 50.29013417187368
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Factor Fields, a novel framework for modeling and representing signals. Factor Fields decomposes a signal into a product of factors, each represented by a classical or neural field representation which operates on transformed input coordinates. This decomposition results in a unified framework that accommodates several recent signal representations including NeRF, Plenoxels, EG3D, Instant-NGP, and TensoRF. Additionally, our framework allows for the creation of powerful new signal representations, such as the "Dictionary Field" (DiF) which is a second contribution of this paper. Our experiments show that DiF leads to improvements in approximation quality, compactness, and training time when compared to previous fast reconstruction methods. Experimentally, our representation achieves better image approximation quality on 2D image regression tasks, higher geometric quality when reconstructing 3D signed distance fields, and higher compactness for radiance field reconstruction tasks. Furthermore, DiF enables generalization to unseen images/3D scenes by sharing bases across signals during training which greatly benefits use cases such as image regression from sparse observations and few-shot radiance field reconstruction.

Related papers

• $R^2$-Mesh: Reinforcement Learning Powered Mesh Reconstruction via Geometry and Appearance Refinement [5.810659946867557]
  Mesh reconstruction based on Neural Radiance Fields (NeRF) is popular in a variety of applications such as computer graphics, virtual reality, and medical imaging. We propose a novel algorithm that progressively generates and optimize meshes from multi-view images. Our method delivers highly competitive and robust performance in both mesh rendering quality and geometric quality.
  arXiv  Detail & Related papers  (2024-08-19T16:33:17Z)
• RaFE: Generative Radiance Fields Restoration [38.602849644666165]
  NeRF (Neural Radiance Fields) has demonstrated tremendous potential in novel view synthesis and 3D reconstruction. Previous methods for NeRF restoration are tailored for specific degradation type, ignoring the generality of restoration. We propose a generic radiance fields restoration pipeline, named RaFE, which applies to various types of degradations.
  arXiv  Detail & Related papers  (2024-04-04T17:59:50Z)
• ResFields: Residual Neural Fields for Spatiotemporal Signals [61.44420761752655]
  ResFields is a novel class of networks specifically designed to effectively represent complex temporal signals. We conduct comprehensive analysis of the properties of ResFields and propose a matrix factorization technique to reduce the number of trainable parameters. We demonstrate the practical utility of ResFields by showcasing its effectiveness in capturing dynamic 3D scenes from sparse RGBD cameras.
  arXiv  Detail & Related papers  (2023-09-06T16:59:36Z)
• Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization [73.3605319281966]
  We propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.
  arXiv  Detail & Related papers  (2023-09-04T10:42:56Z)
• NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies [87.06532943371575]
  We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
  arXiv  Detail & Related papers  (2022-11-25T15:21:45Z)
• PREF: Phasorial Embedding Fields for Compact Neural Representations [54.44527545923917]
  We present a phasorial embedding field emphPREF as a compact representation to facilitate neural signal modeling and reconstruction tasks. Our experiments show PREF-based neural signal processing technique is on par with the state-of-the-art in 2D image completion, 3D SDF surface regression, and 5D radiance field reconstruction.
  arXiv  Detail & Related papers  (2022-05-26T17:43:03Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.