IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

• URL: http://arxiv.org/abs/2302.14683v1
• Date: Tue, 28 Feb 2023 15:51:19 GMT
• Title: IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF
• Authors: Bo Peng, Jun Hu, Jingtao Zhou, Xuan Gao, Juyong Zhang
• Abstract summary: We propose IntrinsicNGP, which can train from scratch and achieve high-fidelity results in few minutes with videos of a human performer. We introduce a continuous and optimizable intrinsic coordinate rather than the original explicit Euclidean coordinate in the hash encoding module of instant-NGP. With this novel intrinsic coordinate, IntrinsicNGP can aggregate inter-frame information for dynamic objects with the help of proxy geometry shapes.
• Score: 27.660829835903424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, many works have been proposed to utilize the neural radiance field for novel view synthesis of human performers. However, most of these methods require hours of training, making them difficult for practical use. To address this challenging problem, we propose IntrinsicNGP, which can train from scratch and achieve high-fidelity results in few minutes with videos of a human performer. To achieve this target, we introduce a continuous and optimizable intrinsic coordinate rather than the original explicit Euclidean coordinate in the hash encoding module of instant-NGP. With this novel intrinsic coordinate, IntrinsicNGP can aggregate inter-frame information for dynamic objects with the help of proxy geometry shapes. Moreover, the results trained with the given rough geometry shapes can be further refined with an optimizable offset field based on the intrinsic coordinate.Extensive experimental results on several datasets demonstrate the effectiveness and efficiency of IntrinsicNGP. We also illustrate our approach's ability to edit the shape of reconstructed subjects.

Related papers

• AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields [14.219495227765671]
  We present AROMA, a framework designed to enhance the modeling of partial differential equations (PDEs) using local neural fields. Our flexible encoder-decoder architecture can obtain smooth latent representations of spatial physical fields from a variety of data types. By employing a diffusion-based formulation, we achieve greater stability and enable longer rollouts compared to conventional MSE training.
  arXiv  Detail & Related papers  (2024-06-04T10:12:09Z)
• From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems [20.006163951844357]
  We propose a simulation-free framework for training neural ordinary differential equations (NODEs) We employ the Fourier analysis to estimate temporal and potential high-order spatial gradients from noisy observational data. Our approach outperforms state-of-the-art methods in terms of training time, dynamics prediction, and robustness.
  arXiv  Detail & Related papers  (2024-05-19T13:15:23Z)
• ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field [89.54363625953044]
  Collaborative Neural Radiance Fields (ColNeRF) is designed to work with sparse input. ColNeRF is capable of capturing richer and more generalized scene representation. Our approach exhibits superiority in fine-tuning towards adapting to new scenes.
  arXiv  Detail & Related papers  (2023-12-14T16:26:46Z)
• INFINITY: Neural Field Modeling for Reynolds-Averaged Navier-Stokes Equations [13.242926257057084]
  INFINITY is a deep learning model that encodes geometric information and physical fields into compact representations. Our framework achieves state-of-the-art performance by accurately inferring physical fields throughout the volume and surface. Our model can correctly predict drag and lift coefficients while adhering to the equations.
  arXiv  Detail & Related papers  (2023-07-25T14:35:55Z)
• GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images [79.39247661907397]
  We introduce an effective framework Generalizable Model-based Neural Radiance Fields to synthesize free-viewpoint images. Specifically, we propose a geometry-guided attention mechanism to register the appearance code from multi-view 2D images to a geometry proxy.
  arXiv  Detail & Related papers  (2023-03-24T03:32:02Z)
• Fast-SNARF: A Fast Deformer for Articulated Neural Fields [92.68788512596254]
  We propose a new articulation module for neural fields, Fast-SNARF, which finds accurate correspondences between canonical space and posed space. Fast-SNARF is a drop-in replacement in to our previous work, SNARF, while significantly improving its computational efficiency. Because learning of deformation maps is a crucial component in many 3D human avatar methods, we believe that this work represents a significant step towards the practical creation of 3D virtual humans.
  arXiv  Detail & Related papers  (2022-11-28T17:55:34Z)
• Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering [139.159534903657]
  We develop a generalizable and efficient Neural Radiance Field (NeRF) pipeline for high-fidelity free-viewpoint human body details. To better tackle self-occlusion, we devise a geometry-guided multi-view feature integration approach. For achieving higher rendering efficiency, we introduce a geometry-guided progressive rendering pipeline.
  arXiv  Detail & Related papers  (2021-12-08T14:42:10Z)
• Deep Magnification-Flexible Upsampling over 3D Point Clouds [103.09504572409449]
  We propose a novel end-to-end learning-based framework to generate dense point clouds. We first formulate the problem explicitly, which boils down to determining the weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted weights as well as the high-order refinements.
  arXiv  Detail & Related papers  (2020-11-25T14:00:18Z)
• Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks [72.62702932371148]
  We introduce a simple autoencoder based on hyperbolic geometry for solving standard collaborative filtering problem. In contrast to many modern deep learning techniques, we build our solution using only a single hidden layer.
  arXiv  Detail & Related papers  (2020-08-15T13:21:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.