Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

• URL: http://arxiv.org/abs/2308.11025v1
• Date: Mon, 21 Aug 2023 20:27:33 GMT
• Title: Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction
• Authors: Sijia Jiang, Jing Hua, Zhizhong Han
• Abstract summary: We introduce neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art.
• Score: 28.910183274743872
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, huge progress has been made on learning neural implicit representations from multi-view images for 3D reconstruction. As an additional input complementing coordinates, using sinusoidal functions as positional encodings plays a key role in revealing high frequency details with coordinate-based neural networks. However, high frequency positional encodings make the optimization unstable, which results in noisy reconstructions and artifacts in empty space. To resolve this issue in a general sense, we introduce to learn neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. Instead of continuous coordinates, we discretize continuous coordinates into discrete coordinates using nearest interpolation among quantized coordinates which are obtained by discretizing the field in an extremely high resolution. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. This significantly reduces the variations in the sample space, and triggers more multi-view consistency constraints on intersections of rays from different views, which enables to infer implicit function in a more effective way. Our quantized coordinates do not bring any computational burden, and can seamlessly work upon the latest methods. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art. Our code is available at https://github.com/MachinePerceptionLab/CQ-NIR.

Related papers

• Fast Point Cloud Geometry Compression with Context-based Residual Coding and INR-based Refinement [19.575833741231953]
  We use the KNN method to determine the neighborhoods of raw surface points. A conditional probability model is adaptive to local geometry, leading to significant rate reduction. We incorporate an implicit neural representation into the refinement layer, allowing the decoder to sample points on the underlying surface at arbitrary densities.
  arXiv  Detail & Related papers  (2024-08-06T05:24:06Z)
• HYVE: Hybrid Vertex Encoder for Neural Distance Fields [9.40036617308303]
  We present a neural-network architecture suitable for accurate encoding of 3D shapes in a single forward pass. Our network is able to output valid signed distance fields without explicit prior knowledge of non-zero distance values or shape occupancy.
  arXiv  Detail & Related papers  (2023-10-10T14:07:37Z)
• Locality-Aware Generalizable Implicit Neural Representation [54.93702310461174]
  Generalizable implicit neural representation (INR) enables a single continuous function to represent multiple data instances. We propose a novel framework for generalizable INR that combines a transformer encoder with a locality-aware INR decoder. Our framework significantly outperforms previous generalizable INRs and validates the usefulness of the locality-aware latents for downstream tasks.
  arXiv  Detail & Related papers  (2023-10-09T11:26:58Z)
• Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis [82.31272171857623]
  We harness low-frequency neural fields to regularize high-frequency neural fields from overfitting. We propose a simple-yet-effective strategy for tuning the frequency to avoid overfitting few-shot inputs.
  arXiv  Detail & Related papers  (2023-03-15T05:15:21Z)
• Neural Fourier Filter Bank [18.52741992605852]
  We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise.
  arXiv  Detail & Related papers  (2022-12-04T03:45:08Z)
• Differentiable Uncalibrated Imaging [25.67247660827913]
  We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles. We apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates. Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration.
  arXiv  Detail & Related papers  (2022-11-18T22:48:09Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Neural Implicit Dictionary via Mixture-of-Expert Training [111.08941206369508]
  We present a generic INR framework that achieves both data and training efficiency by learning a Neural Implicit Dictionary (NID) Our NID assembles a group of coordinate-based Impworks which are tuned to span the desired function space. Our experiments show that, NID can improve reconstruction of 2D images or 3D scenes by 2 orders of magnitude faster with up to 98% less input data.
  arXiv  Detail & Related papers  (2022-07-08T05:07:19Z)
• Variable Bitrate Neural Fields [75.24672452527795]
  We present a dictionary method for compressing feature grids, reducing their memory consumption by up to 100x. We formulate the dictionary optimization as a vector-quantized auto-decoder problem which lets us learn end-to-end discrete neural representations in a space where no direct supervision is available.
  arXiv  Detail & Related papers  (2022-06-15T17:58:34Z)
• Trading Positional Complexity vs. Deepness in Coordinate Networks [33.90893096003318]
  We show that alternative non-Fourier embedding functions can indeed be used for positional encoding. Their performance is entirely determined by a trade-off between the stable rank of the embedded matrix and the distance preservation between embedded coordinates. We argue that employing a more complex positional encoding -- that scales exponentially with the number of modes -- requires only a linear (rather than deep) coordinate function to achieve comparable performance.
  arXiv  Detail & Related papers  (2022-05-18T15:17:09Z)
• Learnable Fourier Features for Multi-DimensionalSpatial Positional Encoding [96.9752763607738]
  We propose a novel positional encoding method based on learnable Fourier features. Our experiments show that our learnable feature representation for multi-dimensional positional encoding outperforms existing methods.
  arXiv  Detail & Related papers  (2021-06-05T04:40:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.