Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE

• URL: http://arxiv.org/abs/2007.04780v1
• Date: Thu, 9 Jul 2020 13:23:15 GMT
• Title: Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE
• Authors: Anna Volokitin, Ertunc Erdil, Neerav Karani, Kerem Can Tezcan, Xiaoran Chen, Luc Van Gool, Ender Konukoglu
• Abstract summary: We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices. We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
• Score: 66.63629641650572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Probabilistic modelling has been an essential tool in medical image analysis, especially for analyzing brain Magnetic Resonance Images (MRI). Recent deep learning techniques for estimating high-dimensional distributions, in particular Variational Autoencoders (VAEs), opened up new avenues for probabilistic modeling. Modelling of volumetric data has remained a challenge, however, because constraints on available computation and training data make it difficult effectively leverage VAEs, which are well-developed for 2D images. We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices. We do so by estimating the sample mean and covariance in the latent space of the 2D model over the slice direction. This combined model lets us sample new coherent stacks of latent variables to decode into slices of a volume. We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy. We demonstrate that our proposed model is competitive in generating high quality volumes at high resolutions according to both traditional metrics and our proposed evaluation.

Related papers

• Introducing 3D Representation for Medical Image Volume-to-Volume Translation via Score Fusion [3.3559609260669303]
  We present Score-Fusion, a novel volumetric translation model that effectively learns 3D representations by ensembling perpendicularly trained 2D diffusion models in score function space. We show that Score-Fusion achieves superior accuracy and volumetric fidelity in 3D medical image super-resolution and modality translation.
  arXiv  Detail & Related papers  (2025-01-13T15:54:21Z)
• Resolution-Robust 3D MRI Reconstruction with 2D Diffusion Priors: Diverse-Resolution Training Outperforms Interpolation [18.917672392645006]
  2D diffusion models trained on 2D slices are starting to be leveraged for 3D MRI reconstruction. Existing methods pertain to a fixed voxel size, and performance degrades when the voxel size is varied. We propose and study several approaches for resolution-robust 3D MRI reconstruction with 2D diffusion priors.
  arXiv  Detail & Related papers  (2024-12-24T18:25:50Z)
• DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models [67.50989119438508]
  We introduce DSplats, a novel method that directly denoises multiview images using Gaussian-based Reconstructors to produce realistic 3D assets. Our experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction.
  arXiv  Detail & Related papers  (2024-12-11T07:32:17Z)
• A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision [65.33043028101471]
  We introduce a diffusion model for Gaussian Splats, SplatDiffusion, to enable generation of three-dimensional structures from single images. Existing methods rely on deterministic, feed-forward predictions, which limit their ability to handle the inherent ambiguity of 3D inference from 2D data.
  arXiv  Detail & Related papers  (2024-12-01T00:29:57Z)
• Latent 3D Brain MRI Counterfactual [25.598362396064367]
  We propose a two-stage method that constructs a Structural Causal Model (SCM) within the latent space. In the first stage, we employ a VQ-VAE to learn a compact embedding of the MRI volume. Subsequently, we integrate our causal model into this latent space and execute a three-step counterfactual procedure.
  arXiv  Detail & Related papers  (2024-09-09T13:15:03Z)
• Inter-slice Super-resolution of Magnetic Resonance Images by Pre-training and Self-supervised Fine-tuning [49.197385954021456]
  In clinical practice, 2D magnetic resonance (MR) sequences are widely adopted. While individual 2D slices can be stacked to form a 3D volume, the relatively large slice spacing can pose challenges for visualization and subsequent analysis tasks. To reduce slice spacing, deep-learning-based super-resolution techniques are widely investigated. Most current solutions require a substantial number of paired high-resolution and low-resolution images for supervised training, which are typically unavailable in real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-10T02:20:26Z)
• Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis [35.45013834475523]
  Cross-modality medical image synthesis is a critical topic and has the potential to facilitate numerous applications in the medical imaging field. Most current medical image synthesis methods rely on generative adversarial networks and suffer from notorious mode collapse and unstable training. We introduce a new paradigm for volumetric medical data synthesis by leveraging 2D backbones and present a diffusion-based framework, Make-A-Volume.
  arXiv  Detail & Related papers  (2023-07-19T16:01:09Z)
• Interpretable 2D Vision Models for 3D Medical Images [47.75089895500738]
  This study proposes a simple approach of adapting 2D networks with an intermediate feature representation for processing 3D images. We show on all 3D MedMNIST datasets as benchmark and two real-world datasets consisting of several hundred high-resolution CT or MRI scans that our approach performs on par with existing methods.
  arXiv  Detail & Related papers  (2023-07-13T08:27:09Z)
• Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models [52.529394863331326]
  We propose a novel approach using two perpendicular pre-trained 2D diffusion models to solve the 3D inverse problem. Our method is highly effective for 3D medical image reconstruction tasks, including MRI Z-axis super-resolution, compressed sensing MRI, and sparse-view CT.
  arXiv  Detail & Related papers  (2023-03-15T08:28:06Z)
• Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation [13.158995287578316]
  We propose a dynamic architecture network named Med-DANet to achieve effective accuracy and efficiency trade-off. For each slice of the input 3D MRI volume, our proposed method learns a slice-specific decision by the Decision Network. Our proposed method achieves comparable or better results than previous state-of-the-art methods for 3D MRI brain tumor segmentation.
  arXiv  Detail & Related papers  (2022-06-14T03:25:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.