3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks

• URL: http://arxiv.org/abs/2110.07766v1
• Date: Thu, 14 Oct 2021 23:05:47 GMT
• Title: 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks
• Authors: Okan Alting\"ovde, Anastasiia Mishchuk, Gulnaz Ganeeva, Emad Oveisi, Cecile Hebert, Pascal Fua
• Abstract summary: We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
• Score: 52.710012864395246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Curvilinear structures frequently appear in microscopy imaging as the object of interest. Crystallographic defects, i.e., dislocations, are one of the curvilinear structures that have been repeatedly investigated under transmission electron microscopy (TEM) and their 3D structural information is of great importance for understanding the properties of materials. 3D information of dislocations is often obtained by tomography which is a cumbersome process since it is required to acquire many images with different tilt angles and similar imaging conditions. Although, alternative stereoscopy methods lower the number of required images to two, they still require human intervention and shape priors for accurate 3D estimation. We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs by utilizing deep convolutional neural networks (CNNs) without making any prior assumption on 3D shapes. In this work, we mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.

Related papers

• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• Using convolutional neural networks for stereological characterization of 3D hetero-aggregates based on synthetic STEM data [0.0]
  A parametric 3D model is presented, from which a wide spectrum of virtual hetero-aggregates can be generated. The virtual structures are passed to a physics-based simulation tool in order to generate virtual scanning transmission electron microscopy (STEM) images. Convolutional neural networks are trained to predict 3D structures of hetero-aggregates from 2D STEM images.
  arXiv  Detail & Related papers  (2023-10-27T22:49:08Z)
• Geometry-Aware Attenuation Learning for Sparse-View CBCT Reconstruction [53.93674177236367]
  Cone Beam Computed Tomography (CBCT) plays a vital role in clinical imaging. Traditional methods typically require hundreds of 2D X-ray projections to reconstruct a high-quality 3D CBCT image. This has led to a growing interest in sparse-view CBCT reconstruction to reduce radiation doses. We introduce a novel geometry-aware encoder-decoder framework to solve this problem.
  arXiv  Detail & Related papers  (2023-03-26T14:38:42Z)
• Enforcing connectivity of 3D linear structures using their 2D projections [54.0598511446694]
  We propose to improve the 3D connectivity of our results by minimizing a sum of topology-aware losses on their 2D projections. This suffices to increase the accuracy and to reduce the annotation effort required to provide the required annotated training data.
  arXiv  Detail & Related papers  (2022-07-14T11:42:18Z)
• Optical Diffraction Tomography based on 3D Physics-Inspired Neural Network (PINN) [0.1310865248866973]
  Optical diffraction tomography (ODT) is an emerging 3D imaging technique that is used for the 3D reconstruction of the refractive index (RI) for semi-transparent samples. Various inverse models have been proposed to reconstruct the 3D RI based on the holographic detection of different samples such as the Born and the Rytov approximations. We propose a different approach where a 3D neural network (NN) is employed. The NN is trained with a cost function derived from a physical model based on the physics of optical wave propagation.
  arXiv  Detail & Related papers  (2022-06-10T17:19:04Z)
• Machine Learning for Detection of 3D Features using sparse X-ray data [6.295613527861694]
  In inertial confinement fusion experiments, the neutron yield and other parameters cannot be completely accounted for with one and two dimensional models. This discrepancy suggests that there are three dimensional effects which may be significant. Sources of these effects include defects in the shells and shell interfaces, the fill tube of the capsule, and the joint feature in double shell targets. We utilize convolutional neural networks to produce different 3D representations of ICF implosions from the experimental data.
  arXiv  Detail & Related papers  (2022-06-02T22:36:54Z)
• Capturing Shape Information with Multi-Scale Topological Loss Terms for 3D Reconstruction [7.323706635751351]
  We propose to complement geometrical shape information by including multi-scale topological features, such as connected components, cycles, and voids, in the reconstruction loss. Our method calculates topological features from 3D volumetric data based on cubical complexes and uses an optimal transport distance to guide the reconstruction process. We demonstrate the utility of our loss by incorporating it into SHAPR, a model for predicting the 3D cell shape of individual cells based on 2D microscopy images.
  arXiv  Detail & Related papers  (2022-03-03T13:18:21Z)
• 3D Structure from 2D Microscopy images using Deep Learning [0.0]
  Recent advances in Artificial Intelligence have been applied to retrieving accurate 3D structures from microscopy images. Herewe present a deep learning solution for reconstructing the protein com-plexes from a number of 2D single molecule localization microscopy images.
  arXiv  Detail & Related papers  (2021-10-14T14:55:41Z)
• CS2-Net: Deep Learning Segmentation of Curvilinear Structures in Medical Imaging [90.78899127463445]
  We propose a generic and unified convolution neural network for the segmentation of curvilinear structures. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder.
  arXiv  Detail & Related papers  (2020-10-15T03:06:37Z)
• Learning Local Neighboring Structure for Robust 3D Shape Representation [143.15904669246697]
  Representation learning for 3D meshes is important in many computer vision and graphics applications. We propose a local structure-aware anisotropic convolutional operation (LSA-Conv) Our model produces significant improvement in 3D shape reconstruction compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-04-21T13:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.