Neural Processing of Tri-Plane Hybrid Neural Fields
- URL: http://arxiv.org/abs/2310.01140v3
- Date: Tue, 30 Jan 2024 11:02:30 GMT
- Title: Neural Processing of Tri-Plane Hybrid Neural Fields
- Authors: Adriano Cardace, Pierluigi Zama Ramirez, Francesco Ballerini, Allan
Zhou, Samuele Salti, Luigi Di Stefano
- Abstract summary: We show that the tri-plane discrete data structure encodes rich information, which can be effectively processed by standard deep-learning machinery.
While processing a field with the same reconstruction quality, we achieve task performance far superior to frameworks that process larges representations.
- Score: 20.78031512517053
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Driven by the appealing properties of neural fields for storing and
communicating 3D data, the problem of directly processing them to address tasks
such as classification and part segmentation has emerged and has been
investigated in recent works. Early approaches employ neural fields
parameterized by shared networks trained on the whole dataset, achieving good
task performance but sacrificing reconstruction quality. To improve the latter,
later methods focus on individual neural fields parameterized as large
Multi-Layer Perceptrons (MLPs), which are, however, challenging to process due
to the high dimensionality of the weight space, intrinsic weight space
symmetries, and sensitivity to random initialization. Hence, results turn out
significantly inferior to those achieved by processing explicit
representations, e.g., point clouds or meshes. In the meantime, hybrid
representations, in particular based on tri-planes, have emerged as a more
effective and efficient alternative to realize neural fields, but their direct
processing has not been investigated yet. In this paper, we show that the
tri-plane discrete data structure encodes rich information, which can be
effectively processed by standard deep-learning machinery. We define an
extensive benchmark covering a diverse set of fields such as occupancy,
signed/unsigned distance, and, for the first time, radiance fields. While
processing a field with the same reconstruction quality, we achieve task
performance far superior to frameworks that process large MLPs and, for the
first time, almost on par with architectures handling explicit representations.
Related papers
- N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures.
We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D.
Our method provides faithful approximations of visibility, depth, and appearance attributes.
arXiv Detail & Related papers (2024-05-25T13:54:34Z) - Reg-NF: Efficient Registration of Implicit Surfaces within Neural Fields [6.949522577812908]
We present Reg-NF, a neural fields-based registration that optimises for the relative 6-DoF transformation between two arbitrary neural fields.
Key components of Reg-NF include a bidirectional registration loss, multi-view surface sampling, and utilisation of volumetric signed distance functions.
arXiv Detail & Related papers (2024-02-15T05:31:03Z) - OReX: Object Reconstruction from Planar Cross-sections Using Neural
Fields [10.862993171454685]
OReX is a method for 3D shape reconstruction from slices alone, featuring a Neural Field gradients as the prior.
A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities.
We offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages.
arXiv Detail & Related papers (2022-11-23T11:44:35Z) - Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond
Algorithms [31.2529724533643]
This work presents the first comprehensive benchmarking study from three under-explored perspectives beyond algorithms.
An analysis on 31 datasets reveals the distinct impacts of data samples.
We achieve a PA-MPJPE of 47.3 mm on the 3DPW test set with a relatively simple model.
arXiv Detail & Related papers (2022-09-21T17:39:53Z) - CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point
Cloud Learning [81.85951026033787]
We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation.
We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration.
The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
arXiv Detail & Related papers (2022-07-31T21:39:15Z) - Inducing Gaussian Process Networks [80.40892394020797]
We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points.
The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains.
We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
arXiv Detail & Related papers (2022-04-21T05:27:09Z) - Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes.
Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
arXiv Detail & Related papers (2021-08-30T19:45:07Z) - Locally Aware Piecewise Transformation Fields for 3D Human Mesh
Registration [67.69257782645789]
We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space.
We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
arXiv Detail & Related papers (2021-04-16T15:16:09Z) - Deep Cellular Recurrent Network for Efficient Analysis of Time-Series
Data with Spatial Information [52.635997570873194]
This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information.
The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
arXiv Detail & Related papers (2021-01-12T20:08:18Z) - Generative Sparse Detection Networks for 3D Single-shot Object Detection [43.91336826079574]
3D object detection has been widely studied due to its potential applicability to many promising areas such as robotics and augmented reality.
Yet, the sparse nature of the 3D data poses unique challenges to this task.
We propose Generative Sparse Detection Network (GSDN), a fully-convolutional single-shot sparse detection network.
arXiv Detail & Related papers (2020-06-22T15:54:24Z) - Dataset Condensation with Gradient Matching [36.14340188365505]
We propose a training set synthesis technique for data-efficient learning, called dataset Condensation, that learns to condense large dataset into a small set of informative synthetic samples for training deep neural networks from scratch.
We rigorously evaluate its performance in several computer vision benchmarks and demonstrate that it significantly outperforms the state-of-the-art methods.
arXiv Detail & Related papers (2020-06-10T16:30:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.