Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior

• URL: http://arxiv.org/abs/2408.08616v1
• Date: Fri, 16 Aug 2024 09:14:12 GMT
• Title: Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior
• Authors: Kyungryun Lee, Won-Ki Jeong,
• Abstract summary: Training a learning-based 3D super-resolution model requires ground truth isotropic volumes and suffers from the curse of dimensionality. Existing methods utilize 2D neural networks to reconstruct each axial slice, eventually piecing together the entire volume. We present a reconstruction framework based on implicit neural representation (INR), which allows 3D coherency even when optimized by independent axial slices.
• Score: 4.1326413814647545
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Analysis and visualization of 3D microscopy images pose challenges due to anisotropic axial resolution, demanding volumetric super-resolution along the axial direction. While training a learning-based 3D super-resolution model seems to be a straightforward solution, it requires ground truth isotropic volumes and suffers from the curse of dimensionality. Therefore, existing methods utilize 2D neural networks to reconstruct each axial slice, eventually piecing together the entire volume. However, reconstructing each slice in the pixel domain fails to give consistent reconstruction in all directions leading to misalignment artifacts. In this work, we present a reconstruction framework based on implicit neural representation (INR), which allows 3D coherency even when optimized by independent axial slices in a batch-wise manner. Our method optimizes a continuous volumetric representation from low-resolution axial slices, using a 2D diffusion prior trained on high-resolution lateral slices without requiring isotropic volumes. Through experiments on real and synthetic anisotropic microscopy images, we demonstrate that our method surpasses other state-of-the-art reconstruction methods. The source code is available on GitHub: https://github.com/hvcl/INR-diffusion.

Related papers

• From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task [19.56372155146739]
  We present a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution. Our results on two publicly available focused ion beam scanning electron microscopy (FIB-SEM) datasets demonstrate the robustness and practical applicability of our framework.
  arXiv  Detail & Related papers  (2024-11-25T09:12:55Z)
• MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images [13.255044855902408]
  We present MV2Cyl, a novel method for reconstructing 3D from 2D multi-view images. We achieve the optimal reconstruction result with the best accuracy in 2D sketch and extrude parameter estimation.
  arXiv  Detail & Related papers  (2024-06-16T08:54:38Z)
• What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs [82.3936309001633]
  3D-aware Generative Adversarial Networks (GANs) have shown remarkable progress in learning to generate multi-view-consistent images and 3D geometries. Yet, the significant memory and computational costs of dense sampling in volume rendering have forced 3D GANs to adopt patch-based training or employ low-resolution rendering with post-processing 2D super resolution. We propose techniques to scale neural volume rendering to the much higher resolution of native 2D images, thereby resolving fine-grained 3D geometry with unprecedented detail.
  arXiv  Detail & Related papers  (2024-01-04T18:50:38Z)
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D [88.66678730537777]
  We present StableDreamer, a methodology incorporating three advances. First, we formalize the equivalence of the SDS generative prior and a simple supervised L2 reconstruction loss. Second, our analysis shows that while image-space diffusion contributes to geometric precision, latent-space diffusion is crucial for vivid color rendition.
  arXiv  Detail & Related papers  (2023-12-02T02:27:58Z)
• Passive superresolution imaging of incoherent objects [63.942632088208505]
  Method consists of measuring the field's spatial mode components in the image plane in the overcomplete basis of Hermite-Gaussian modes and their superpositions. Deep neural network is used to reconstruct the object from these measurements.
  arXiv  Detail & Related papers  (2023-04-19T15:53:09Z)
• Fast 3D Volumetric Image Reconstruction from 2D MRI Slices by Parallel Processing [1.7778609937758323]
  Methods for virtual three-dimensional (3D) reconstruction from a single sequence of two-dimensional (2D) slices of MR images of a human spine and brain are proposed. Our approach helps in preserving the edges, shape, size, as well as the internal tissue structures of the object being captured. To the best of our knowledge it is a first of its kind approach based on kriging and multiprocessing for 3D reconstruction from 2D slices.
  arXiv  Detail & Related papers  (2023-03-16T17:39:11Z)
• Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models [52.529394863331326]
  We propose a novel approach using two perpendicular pre-trained 2D diffusion models to solve the 3D inverse problem. Our method is highly effective for 3D medical image reconstruction tasks, including MRI Z-axis super-resolution, compressed sensing MRI, and sparse-view CT.
  arXiv  Detail & Related papers  (2023-03-15T08:28:06Z)
• Neural Volume Super-Resolution [49.879789224455436]
  We propose a neural super-resolution network that operates directly on the volumetric representation of the scene. To realize our method, we devise a novel 3D representation that hinges on multiple 2D feature planes. We validate the proposed method by super-resolving multi-view consistent views on a diverse set of unseen 3D scenes.
  arXiv  Detail & Related papers  (2022-12-09T04:54:13Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Detailed 3D Human Body Reconstruction from Multi-view Images Combining Voxel Super-Resolution and Learned Implicit Representation [12.459968574683625]
  We propose a coarse-to-fine method to reconstruct a detailed 3D human body from multi-view images. The coarse 3D models are estimated by learning an implicit representation based on multi-scale features. The refined detailed 3D human body models can be produced by the voxel super-resolution which can preserve the details.
  arXiv  Detail & Related papers  (2020-12-11T08:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.