OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields

• URL: http://arxiv.org/abs/2211.12886v3
• Date: Sun, 2 Apr 2023 09:31:02 GMT
• Title: OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields
• Authors: Haim Sawdayee, Amir Vaxman, Amit H. Bermano
• Abstract summary: OReX is a method for 3D shape reconstruction from slices alone, featuring a Neural Field gradients as the prior. A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities. We offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages.
• Score: 10.862993171454685
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reconstructing 3D shapes from planar cross-sections is a challenge inspired by downstream applications like medical imaging and geographic informatics. The input is an in/out indicator function fully defined on a sparse collection of planes in space, and the output is an interpolation of the indicator function to the entire volume. Previous works addressing this sparse and ill-posed problem either produce low quality results, or rely on additional priors such as target topology, appearance information, or input normal directions. In this paper, we present OReX, a method for 3D shape reconstruction from slices alone, featuring a Neural Field as the interpolation prior. A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities. The main challenge for this approach is high-frequency details, as the neural prior is overly smoothing. To alleviate this, we offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages. In addition, we identify and analyze a ripple-like effect stemming from the mesh extraction step. We mitigate it by regularizing the spatial gradients of the indicator function around input in/out boundaries during network training, tackling the problem at the root. Through extensive qualitative and quantitative experimentation, we demonstrate our method is robust, accurate, and scales well with the size of the input. We report state-of-the-art results compared to previous approaches and recent potential solutions, and demonstrate the benefit of our individual contributions through analysis and ablation studies.

Related papers

• PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks [30.701422594374456]
  We present a framework for preimage abstraction that produces under- and over-approximations of any polyhedral output set. We evaluate our method on a range of tasks, demonstrating significant improvement in efficiency and scalability to high-input-dimensional image classification tasks.
  arXiv  Detail & Related papers  (2024-08-17T17:24:47Z)
• Unsupervised Occupancy Learning from Sparse Point Cloud [8.732260277121547]
  Implicit Neural Representations have gained prominence as a powerful framework for capturing complex data modalities. In this paper, we propose a method to infer occupancy fields instead of Neural Signed Distance Functions. We highlight its capacity to improve implicit shape inference with respect to baselines and the state-of-the-art using synthetic and real data.
  arXiv  Detail & Related papers  (2024-04-03T14:05:39Z)
• NeuralGF: Unsupervised Point Normal Estimation by Learning Neural Gradient Function [55.86697795177619]
  Normal estimation for 3D point clouds is a fundamental task in 3D geometry processing. We introduce a new paradigm for learning neural gradient functions, which encourages the neural network to fit the input point clouds. Our excellent results on widely used benchmarks demonstrate that our method can learn more accurate normals for both unoriented and oriented normal estimation tasks.
  arXiv  Detail & Related papers  (2023-11-01T09:25:29Z)
• On the ISS Property of the Gradient Flow for Single Hidden-Layer Neural Networks with Linear Activations [0.0]
  We investigate the effects of overfitting on the robustness of gradient-descent training when subject to uncertainty on the gradient estimation. We show that the general overparametrized formulation introduces a set of spurious equilibria which lay outside the set where the loss function is minimized.
  arXiv  Detail & Related papers  (2023-05-17T02:26:34Z)
• Towards Confidence-guided Shape Completion for Robotic Applications [6.940242990198]
  Deep learning has begun taking traction as effective means of inferring a complete 3D object representation from partial visual data. We propose an object shape completion method based on an implicit 3D representation providing a confidence value for each reconstructed point. We experimentally validate our approach by comparing reconstructed shapes with ground truths, and by deploying our shape completion algorithm in a robotic grasping pipeline.
  arXiv  Detail & Related papers  (2022-09-09T13:48:24Z)
• Learned Vertex Descent: A New Direction for 3D Human Model Fitting [64.04726230507258]
  We propose a novel optimization-based paradigm for 3D human model fitting on images and scans. Our approach is able to capture the underlying body of clothed people with very different body shapes, achieving a significant improvement compared to state-of-the-art. LVD is also applicable to 3D model fitting of humans and hands, for which we show a significant improvement to the SOTA with a much simpler and faster method.
  arXiv  Detail & Related papers  (2022-05-12T17:55:51Z)
• Revisiting Point Cloud Simplification: A Learnable Feature Preserving Approach [57.67932970472768]
  Mesh and Point Cloud simplification methods aim to reduce the complexity of 3D models while retaining visual quality and relevant salient features. We propose a fast point cloud simplification method by learning to sample salient points. The proposed method relies on a graph neural network architecture trained to select an arbitrary, user-defined, number of points from the input space and to re-arrange their positions so as to minimize the visual perception error.
  arXiv  Detail & Related papers  (2021-09-30T10:23:55Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration [67.69257782645789]
  We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
  arXiv  Detail & Related papers  (2021-04-16T15:16:09Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.