Neural rendering enables dynamic tomography

• URL: http://arxiv.org/abs/2410.20558v1
• Date: Sun, 27 Oct 2024 19:18:20 GMT
• Title: Neural rendering enables dynamic tomography
• Authors: Ivan Grega, William F. Whitney, Vikram S. Deshpande,
• Abstract summary: We propose that neural rendering tools can be used to drive the paradigm shift to enable 3d reconstruction during dynamic events. We demonstrate that neural fields can reconstruct data modalities more efficiently than conventional reconstruction methods.
• Score: 1.1624569521079426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interrupted X-ray computed tomography (X-CT) has been the common way to observe the deformation of materials during an experiment. While this approach is effective for quasi-static experiments, it has never been possible to reconstruct a full 3d tomography during a dynamic experiment which cannot be interrupted. In this work, we propose that neural rendering tools can be used to drive the paradigm shift to enable 3d reconstruction during dynamic events. First, we derive theoretical results to support the selection of projections angles. Via a combination of synthetic and experimental data, we demonstrate that neural radiance fields can reconstruct data modalities of interest more efficiently than conventional reconstruction methods. Finally, we develop a spatio-temporal model with spline-based deformation field and demonstrate that such model can reconstruct the spatio-temporal deformation of lattice samples in real-world experiments.

Related papers

• DYRECT Computed Tomography: DYnamic Reconstruction of Events on a Continuous Timescale [0.19249183980865092]
  Time-resolved high-resolution X-ray Computed Tomography (4D $mu$CT) is an imaging technique that offers insight into the evolution of dynamic processes inside materials that are opaque to visible light. Conventional tomographic reconstruction techniques are based on recording a sequence of 3D images that represent the sample state at different moments in time. This frame-based approach limits the temporal resolution compared to dynamic radiography experiments due to the time needed to make CT scans. Our proposed 4D $mu$CT reconstruction technique, named DYRECT, estimates individual attenuation evolution profiles for each position in the sample.
  arXiv  Detail & Related papers  (2024-11-15T14:21:46Z)
• Data-driven discovery of mechanical models directly from MRI spectral data [0.0]
  We propose a reconstruction framework for data-driven discovery of dynamical models from experimentally obtained undersampled MRI spectral data. The proposed framework combines this method with data-driven discovery of interpretable models using Sparse Identification of Non-linear Dynamics (SINDy) It is successfully validated using spectral data of a dynamic phantom gathered on a clinical MRI scanner.
  arXiv  Detail & Related papers  (2024-11-11T13:05:29Z)
• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• Optical Diffraction Tomography based on 3D Physics-Inspired Neural Network (PINN) [0.1310865248866973]
  Optical diffraction tomography (ODT) is an emerging 3D imaging technique that is used for the 3D reconstruction of the refractive index (RI) for semi-transparent samples. Various inverse models have been proposed to reconstruct the 3D RI based on the holographic detection of different samples such as the Born and the Rytov approximations. We propose a different approach where a 3D neural network (NN) is employed. The NN is trained with a cost function derived from a physical model based on the physics of optical wave propagation.
  arXiv  Detail & Related papers  (2022-06-10T17:19:04Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Enhancement of Novel View Synthesis Using Omnidirectional Image Completion [61.78187618370681]
  We present a method for synthesizing novel views from a single 360-degree RGB-D image based on the neural radiance field (NeRF) Experiments demonstrated that the proposed method can synthesize plausible novel views while preserving the features of the scene for both artificial and real-world data.
  arXiv  Detail & Related papers  (2022-03-18T13:49:25Z)
• Neural Computed Tomography [1.7188280334580197]
  Motion during acquisition of a set of projections can lead to significant motion artifacts in computed tomography reconstructions. We propose a novel reconstruction framework, NeuralCT, to generate time-resolved images free from motion artifacts.
  arXiv  Detail & Related papers  (2022-01-17T18:50:58Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples [1.112751058850223]
  We develop a new model-based data normalization pre-processing procedure for homogenizing the sample contrast. We demonstrate this framework's capabilities on experimental measurements of epithelial buccal cells and Caenorhabditis elegans worms.
  arXiv  Detail & Related papers  (2021-03-29T17:43:56Z)
• Probabilistic 3D surface reconstruction from sparse MRI information [58.14653650521129]
  We present a novel probabilistic deep learning approach for concurrent 3D surface reconstruction from sparse 2D MR image data and aleatoric uncertainty prediction. Our method is capable of reconstructing large surface meshes from three quasi-orthogonal MR imaging slices from limited training sets.
  arXiv  Detail & Related papers  (2020-10-05T14:18:52Z)
• Real-time sparse-sampled Ptychographic imaging through deep neural networks [3.3351024234383946]
  A ptychography reconstruction is achieved by means of solving a complex inverse problem that imposes constraints both on the acquisition and on the analysis of the data. We propose PtychoNN, a novel approach to solve the ptychography reconstruction problem based on deep convolutional neural networks.
  arXiv  Detail & Related papers  (2020-04-15T23:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.